Overview

1. Introduction to LinkedIn ads as survey recruitment method
2. Recruiting via LinkedIn – a step-by-step guide
   1. LinkedIn Campaign Manager
      1. Personal LinkedIn account
      2. LinkedIn ad account
      3. LinkedIn page
   2. Setting up an ad campaign
      1. Campaign level
      2. Ad set level
      3. Ad level
3. Ethical considerations
4. Application Example
   1. Ad campaign details
      1. Ad creatives
      2. Campaign setup
   2. Campaign results
   3. Sample composition
5. Summary and key takeaways
6. Further reading

Introduction to LinkedIn ads as survey recruitment method

Social media platforms have become an increasingly common tool for survey participant recruitment, allowing researchers to reach large populations quickly, cost-effectively, and with fine-grained targeting options. Most existing methodological work in this area, however, has focused on general-purpose platforms such as Facebook and Instagram. Systematic reviews of social media recruitment consistently show that these platforms dominate the field, whereas LinkedIn appears only sporadically and is rarely the central focus of empirical or methodological studies (e.g., Zindel 2023; Jones et al. 2023). This imbalance is striking, because LinkedIn offers a professionalized user base, rich profile information, and targeting parameters that are directly relevant to research on labor market and workplace related topics.

LinkedIn is a professional networking platform where people connect, share content, and participate in industry-specific communities. With more than one billion registered members worldwide (LinkedIn 2025), it has become a central hub for working professionals, job seekers, and industry leaders. Because LinkedIn is explicitly work-oriented – and because user profiles contain rich educational and occupational information – it offers considerable potential for social science research. The platform mirrors real-world professional networks and career trajectories, making it particularly interesting for studies on labor market behavior, employment trends, and professional attitudes. A central strength of LinkedIn is the ability to reach narrowly defined populations based on industry, occupation, seniority, education, or geographic region. These segments are often difficult or impossible to access through traditional recruitment channels or general social media platforms. The granularity of profile information and targeting options enables researchers to sample very specific occupational groups or sectors, which is valuable for both descriptive and analytical research on employment and organizations. But next to the numerous possibilities of survey recruitment on LinkedIn, researchers must evaluate potential challenges regarding data protection, representativeness, and further potential bias.

To date, researchers use LinkedIn mostly either as an object of study – for example, to examine recruitment practices in labor markets, employer branding, or self-presentation strategies (e.g., Hosain & Liu 2021; Caers & Castelyns 2010) – or as a data source for analyzing profile characteristics and network structures (e.g., Banerji & Reimer 2019; Zide et al. 2014). Some more recent work has begun to use LinkedIn to complement survey data, for instance by linking respondents’ answers to information from their LinkedIn profiles or from organizational pages (e.g., Al Baghal et al. 2024). These studies illustrate that LinkedIn can provide rich contextual information on careers and organizations, but they typically focus on the use of existing profile data rather than on recruitment processes.

Researchers have also started experimenting with LinkedIn as a direct recruitment tool for survey-based studies. Documented approaches include promoting surveys through public (e.g., Stokes et al. 2017; Keemink et al. 2025) or direct messages and personalized outreach (Griffiths et al. 2025; Kozłowski et al. 2021). However, methodological guidance on how to employ LinkedIn in a systematic and reproducible way remains limited. In particular, paid LinkedIn advertisements are still rarely documented in the academic literature. Existing accounts tend to be reflective case studies rather than detailed tutorials, and they often provide only high-level descriptions of campaign configurations and outcomes (e.g., Kohl et al., 2023).

Recruiting via LinkedIn – a step-by-step guide

This section outlines a practical workflow for designing and implementing LinkedIn advertising campaigns for survey recruitment.

LinkedIn campaign manager

Before launching any advertisements, several prerequisites must be met. Researchers require:

1. a personal LinkedIn account,
2. a LinkedIn ad account, and
3. a LinkedIn page.

Each component serves a distinct purpose in the LinkedIn advertising ecosystem. The creation of all three components is free of charge.

Personal LinkedIn account

Advertising on LinkedIn requires an active personal account associated with a real individual (i.e., a verifiable professional identity). During registration, the user must provide standard information such as name, location, job title, and current employer. For authenticity and compliance with LinkedIn’s advertising policies, the account should not be newly created: it must be at least 24 hours old, have at least one verified connection, and represent a verifiable professional identity. These criteria ensure that advertisements are traceable to legitimate users rather than anonymous entities or automated accounts. Importantly, these personal accounts are used for access and accountability within LinkedIn’s advertising system, but they are not displayed as the public-facing identity of the ads. Instead, the advertisements are shown as being placed by the LinkedIn Page (described below), meaning it is not externally visible which individual account initiated or manages a given campaign.

LinkedIn ad account

Once a personal profile exists, an advertising account can be established via the LinkedIn Advertise platform. Selecting “Create ad” or “Get Started” initiates the process (see Figure 1).

Figure 1: Starting page of LinkedIn Advertise website.

_{^{Source: LinkedIn Advertise}}

The platform then prompts the user to assign an account name and link the account to a LinkedIn Page (see Figure 2). If no appropriate institutional page is available, a new one must be created.

Figure 2: LinkedIn ads account set up.

_{^{Source: LinkedIn Campaign Manager}}

After this step, the ad account provides access to campaign creation, audience definition, and budgeting functions.

LinkedIn page

A LinkedIn page functions as the institutional or organizational identity under which advertisements are displayed. Depending on the project’s context, the page may represent:

• a company or organization (e.g., a research institute),
• a showcase page (a subpage linked to a main institutional account), or
• an educational institution (e.g., an University).

Unlike platforms such as Facebook (Meta), LinkedIn does not currently offer a dedicated project or educational page type, meaning researchers must register under one of the existing categories. Each page requires basic information such as a name, a public URL, the industry, and the size of the organization (see Figure 3). In addition, it is possible to indicate further information (i.e., logo and a short organizational slogan). For research recruitment, using an institutional page (for example, a university department or research group) can increase credibility and transparency in the eyes of potential participants.

Figure 3: Set up LinkedIn page – example: educational institute.

_{^{Source: LinkedIn Campaign Manager}}

Setting up an ad campaign

The LinkedIn advertising system follows a three-tier hierarchical structure:

Figure 4: LinkedIn advertising system

_{^{Source: Own illustration.}}

Each tier governs a different aspect of the set-up, from defining overall objectives to specifying detailed audience characteristics and ad creatives. Understanding this structure is essential for organizing an effective and transparent recruitment effort.

Campaign level

At the highest level, the campaign establishes the overarching goal and organizational framework of the recruitment project. LinkedIn offers three types of objectives (see Figure 5):

• Awareness (increasing visibility),
• Consideration (encouraging engagement), and
• Conversion (prompting specific actions).

For survey recruitment, the “Website visits” option under the Consideration category is generally most suitable, as it directs users to an external landing page, typically the survey entry point.

Figure 5: Choosing a campaign objective

_{^{Source: LinkedIn Campaign Manager}}

Once the campaign objective has been chosen, LinkedIn prompts the user to select a campaign type. Two options are available: a manual setup (“Classic”) and an AI-assisted setup (“Accelerate”) (see Figure 6). While the latter automatically adjusts parameters to optimize engagement, we strongly recommend avoiding it. Algorithmic optimization introduces unknown and uncontrolled sources of bias into audience selection, thereby comprising the transparency and replicability required in scientific research. Manual configuration ensures that all targeting and budgeting decisions remain under the advertiser’s, that is, the researcher’s control and can be documented in detail.

Figure 6: Selecting a campaign type

_{^{Source: LinkedIn Campaign Manager}}

After selecting the campaign type, researchers must determine the ad creation process, that is, how individual advertisement will be assembled and optimized within the campaign. Again, LinkedIn offers two routes: Flexible ad creation and standard ad creation (see Figure 7). In the flexible ad creation mode, the platform automatically generates and tests different combinations of ad elements (e.g., images, headlines, and descriptions) and allocates the budget to those combinations that achieve the best predicted performance. This process resembles A/B testing but is fully automated. Although potentially efficient, this approach lacks transparency. The platform does not disclose the criteria used to define “best performance” nor how these predictions are generated. Consequently, it introduces algorithmic bias and variation that are not under research control. In research contexts, the standard ad creation mode is clearly preferable. This mode requires researchers to manually configure each ad creatively. All text and visual elements, as well as audience and delivery settings, are explicitly defined by the researchers. This manual process ensures that recruitment materials remain consistent and that all campaign parameters are fully reproducible and can be reported in a methods section.

Figure 7: Selecting an ad creation process

_{^{Source: LinkedIn Campaign Manager}}

After determining the creation route, the campaign configuration also includes the definition of schedule and budget parameters (see Figure 8). Researchers may specify:

• a start and end date for the campaign,
• a total (lifetime) budget, and
• whether budget optimization across campaigns should be enabled.

Again, we recommend disabling all forms of automated budget optimization. Although these features can improve marketing efficiency by reallocating funds to higher-performing ads, they rely on algorithmic assessments that are not accessible to researchers and could bias the recruited sample in unknown and unobservable ways.

Figure 8: Campaign details

_{^{Source: LinkedIn Campaign Manager}}

Ad set level

At ad set level, the researcher defines the target population, that is, the group of LinkedIn users who will be exposed to the survey advertisement. LinkedIn provides numerous targeting attributes derived from the members’ professional profiles, company pages, and inferred behavioral or interest signals (see Figure 9). Table 1 provides a comprehensive overview of the main attribute categories and typical options within each. Where known, information on how LinkedIn obtains or infers that information is indicated.

Table 1: LinkedIn Attributes and Data Sources

_{^{Note: Data availability and functionality may vary by region due to privacy regulations, particularly with the Europeam Economic Area and Switzerland.}}

Figure 9: Definition of target audience

_{^{Source: LinkedIn Campaign Manager}}

As seen in Table 1, researchers have various options for target parameters. Therefore, is important that targeting parameters must never be used to discriminate against individuals based on protected characteristics such as gender, age, or actual or perceived race or ethnicity. Researchers should also comply with platform-specific restrictions and regional regulations that limit the use of sensitive targeting criteria. The option “enable audience expansion” should be deactivated (see Figure 9). This algorithmic extension may deliver ads to users outside the intended target population and thus undermine the transparency and controllability of the sample frame. LinkedIn’s Campaign Manager also displays an estimated performance forecast, showing projected target audience size, impressions, clicks, and costs (see Figure 10). These values can assist in budget planning but are based on proprietary models and should be interpreted as approximate indicators only, not as precise predictions.

Figure 10: Forecasted results

_{^{Source: LinkedIn Campaign Manager}}

Once the campaign objectives and audience have been defined, the next step is the selection of an appropriate advertisement format (see Figure 11). LinkedIn currently offers several formats that differ in visual layout, degree of interactivity, and expected engagement levels. The choice of format should align with the overall research design, recruitment objectives, and desired intensity of participation involvement. Available formats include, among other:

• Single image ads: static image with headline and text,
• Carousel ads: multiple scrollable image cards that allow the presentation of different aspects or topics within one advertisement, and
• Video ads: short clip (15-30 seconds) combining visuals and text to communicate information dynamically.

For most survey recruitment, these formats are generally the most suitable due to their high visibility and clear message presentation.

Figure 11: Ad format

_{^{Source: LinkedIn Campaign Manager}}

The integration of URL tracking parameters (e.g., UTM codes) into the destination link is recommended to enable transparent performance monitoring (see Figure 12). A typical example would be:

utm_source=[SOCIAL MEDIA PLATFORM]&utm_medium=[PAYMENT EVENT]
&utm_campaign=[CAMPAIGN NAME]&utm_content=[AD ID].

This method complies with data protection requirements and does not collect personal information. It simply allows researchers to distinguish traffic from different platforms or creatives in their web analytics.

Figure 12: URL tracking parameters

_{^{Source: LinkedIn Campaign Manager}}

Ad placement is then specified (see Figure 13). The LinkedIn feed should be selected as the primary placement to ensure visibility in a professional context. The LinkedIn Audience Network, which extends ad delivery to partner websites and apps, should be deactivated in research campaigns to maintain control and transparency regarding where exactly recruitment occurs.

Figure 13: Placements option

_{^{Source: LinkedIn Campaign Manager}}

The budgeting model determines how funds are distributed over time (Figure 14). LinkedIn provides three options:

• Daily budget: specifies maximum daily expenditure, ensuring consistent ad delivery,
• Lifetime budget: sets a total expenditure limit for the campaign period, and
• Combined budget: applies both daily and lifetime limits for maximum control.

Short-term campaigns benefit from daily budgets to maintain stable exposure, while longer recruitment periods may require lifetime or combined budget settings. Key performance metrics such as impressions, click-through rate (CTR), and cost per click (CPC) should be monitored throughout the campaign and reported alongside sample characteristics.

Figure 14: Budget and schedule options

_{^{Source: LinkedIn Campaign Manager}}

The bidding strategy determines how LinkedIn allocates ads in its auctions system (Figure 15). LinkedIn currently offers three bid types:

• Maximum delivery: an automated bidding option where LinkedIn’s system sets the bid with the goal of spending the full budget and maximizing the selected key result,
• Cost cap: an automated option where LinkedIn sets bids while trying to keep the average cost per key result close to a user-specified target (e.g., CPC or CPM), and
• Manual bidding: a more hands-on option where advertisers set an explicit bid value that is used in the ad auction.

From a research perspective, there is a trade-off between cost-efficiency and methodological control. Fully automated options such as maximum delivery can be convenient and may help use the budget efficiently, but they also shift additional decision-making to opaque algorithms. Where fine-grained cost control and transparency are priorities, cost cap or even manual bidding may be preferable, provided that campaigns are monitored closely, and bid levels are adjusted if necessary. Regardless of the chosen strategy, researchers should document the bid type and any changes during fieldwork so that recruitment conditions can be accurately reported.

Figure 15: Bidding options

_{^{Source: LinkedIn Campaign Manager}}

Although LinkedIn supports conversion tracking via tracking pixels (see Figure 16), this method is ethically and legally problematic in research contexts. Tracking pixels typically collect behavioral data across websites (e.g., page views, time spent, subsequent actions) and can be used to link on-platform interactions with off-platform behavior. Under data protection frameworks such as the General Data Protection Regulation (GDPR), such tracking requires a valid legal basis and, in most cases, informed consent: participants must be clearly informed before any tracking takes place about what data are collected, for what purpose, on which legal basis, how long they are stored, and with whom they may be shared. Only if users have actively agreed (e.g., via a consent banner or consent form on the landing page) may tracking pixels be activated. However, for scientific campaigns, this form of tracking is usually difficult to justify. Social media platforms and advertising networks have repeatedly been affected by data breaches and other security incidents (e.g., Das 2023; Feathers et al. 2022; Fondrie-Teitler et al. 2022). When third-party tracking is enabled, additional actors gain access to behavioral data and identifiers, increasing the potential attack surface. Even if the probability of a breach is difficult to quantify, the possible consequences for participants – for example, the unintended disclosure of sensitive behavioral patterns or re-identification risks when log data are leaked – can be substantial. From a risk-minimization perspective, avoiding non-essential tracking is therefore a central protective measure. We therefore recommend that researchers do not use LinkedIn’s conversion tracking or similar third-party tracking pixels for recruitment. Instead, evaluation should rely on aggregate campaign statistics (impressions, clicks, CPC) provided by the platform, combined with survey-based indicators (e.g., completion rates and a self-reported recruitment channel question on the first page of the questionnaire, asking participants how they became aware of the study).

Figure 16: Conversion tracking

_{^{Source: LinkedIn Campaign Manager}}

Ad level

At the ad level, the creative elements that LinkedIn users are exposed to are defined. The design of these elements can draw on established findings from research on invitation letters in survey methodology. In line with this literature, ad content should be convincing and informative, and should convey a clear perceived leverage for participation, for example by highlighting the social value of the study, the relevance of the topic for the target population, or the opportunity to contribute one’s perspective (e.g., Dillman et al. 2014 ; Groves et al. 2000). On LinkedIn, as on other social media platforms, an ad competes with a large volume of other digital content. Consequently, the ad must first attract attention within a very short time frame and then provide sufficient relevance and clarity to motivate users to click on the ad and proceed to the survey landing page. The introductory text (Figure 17) – typically the most prominent text element in the ad – should succinctly communicate the purpose of the study and its relevance for the targeted audience. A concise structure is advisable: one sentence specifying what the study is about and who it addresses, followed by a sentence that conveys why participation is meaningful (e.g., expected benefits, contribution to knowledge, or policy relevance).

Figure 17: Ad design - Text and media input

_{^{Source: LinkedIn Campaign Manager}}

The visual content serves as the primary attention grabber and can function as an additional targeting device (Figure 17). On LinkedIn, visuals are often embedded in a professional and organizational context, for example by depicting typical work situations (e.g., team meetings, hybrid collaboration, industry-specific workplaces), abstract graphics related to the topic (e.g., charts, icons for digitalization or skills), or neutral employer-branding style imagery. When these visuals are clearly aligned with a specific survey topic – for instance, leadership and management practices, remote work arrangements, skills development, or employee well-being – they can increase click-through rates and the number of completed questionnaires, because they immediately signal topical relevance to certain occupational groups or sectors. At the same time, these visuals may amplify self-selection on work-related attitudes or experiences (e.g., job satisfaction, perceived stress, views on organizational change), as they are particularly salient for individuals with above-average involvement or strong opinions (Donzowa et al. 2025; Zindel et al., 2025). This implies a trade-off: if recruitment within a narrowly defined topic area is prioritized, topic-specific professional visuals are advantageous; if concerns about attitudinal bias are central, more neutral or general workplace imagery may be preferable. In addition to the visual itself, LinkedIn offers the option to set an image alt text (Figure 18). This alt text enhances accessibility for users who rely on screen readers and is displayed if the image cannot be rendered. From a survey recruitment perspective, the alt text should provide a concise, neutral description of what is shown in the image and, where appropriate, can briefly restate the topic of the study (e.g., “Illustration of professionals discussing climate policies in an office setting”). Overly promotional language or keyword stuffing should be avoided. The goal is to ensure that the meaning of the visual remains understandable even in the absence of the actual image and that the alt text is consistent with the overall framing of the study (cf. [World Wide Web Consortium [W3C], 2025[(https://www.w3.org/TR/WCAG21/)]).

Figure 18: Ad design - Further options

_{^{Source: LinkedIn Campaign Manager}}

The destination URL is the link to the survey website and leads directly to the survey entry page, without intermediary pages or redirects. This entry page must present the informed consent statement and essential information on data protection, study sponsors, and contact details before data collection begins. From a user-experience perspective, the number of steps between the ad (i.e., the survey invitation) and the first survey question should be minimized, as each additional step potentially increases drop-off probabilities (cf. Tourangeau et al. 2013). Besides the introductory text and the image, LinkedIn ads typically include a headline and an optional ad description. The headline is usually displayed in a highly prominent position and should therefore be short, specific, and benefit oriented. For survey recruitment, headlines that clearly state the topic and target group (e.g., “Short survey on remote work for HR professionals”) are generally preferable to vague or purely promotional formulations. The ad description provides additional information to people who see the ad. However, this field is not visible in most scenarios and will only appear for a small portion of LinkedIn members, for example in certain placements on the LinkedIn Audience Network (if enabled). Accordingly, essential information about the study should not be placed exclusively in the description but should already be contained in the introductory text and headline. The description can be used as a secondary field to elaborate on the headline, clarify what happens after the click, indicate approximate survey length, or highlight a specific leverage (e.g., “10-minute academic survey, anonymous, results will inform future workplace policies”). Given that descriptions may be truncated or omitted depending on device and placement, key information should appear at the beginning of the text and should not be critical for understanding the study. The call-to-action (CTA) button should be consistent with the overall framing of the ad and align with user expectations on a professional platform. LinkedIn offers several standard CTA options, including “Learn more”, “Sign up”, “Register”, and others. For survey recruitment, a CTA such as “Learn more” is typically most appropriate, as it signals low-threshold, information-oriented action compatible with participation in a research study. In settings where the survey is embedded in a broader program or panel, “Sign up” or “Register” may also be suitable.

Figure 19: Ad preview

_{^{Source: LinkedIn Campaign Manager. Image by Africa Studio via stock.adobe.com.}}

Prior to launch, the preview function in Campaign Manager should be used to verify appearance across devices and placements (see Figure 19). Across all ad components, internal consistency is crucial. The introductory text, headline, description, visual elements (including alt text), destination URL, and CTA should convey a coherent message and set realistic expectations about the survey experience. A consistent, methodologically informed ad design that is at the same time adapted to platform-specific norms on LinkedIn potentially increases the likelihood of attracting suitable respondents rather than merely generating clicks.

Ethical considerations

When recruiting survey participants via social media, particular attention should be paid to ethical and data protection issues. Most important, all procedures must comply with applicable data protection frameworks (e.g., GDPR), including a clear specification of the legal basis for processing, information on storage periods, and contact details of the responsible data controller and, where applicable, the data protection officer. Moreover, recruitment messages and landing pages should be transparent about the study’s purpose, the institutions involved, and the intended use of the data. Participants should be informed about their data protection rights (e.g., rights of access, rectification, erasure, restriction of processing, and complaint) in a concise and comprehensible way. Where surveys are anonymous, this should be explicitly communicated and technically ensured, for instance by avoiding the collection of directly identifying information and by separating contact data from survey responses if recontact is planned. Where anonymity cannot be fully guaranteed – for example because organization-level or very narrow professional targeting makes the population of potential invitees very small – this limitation should be stated clearly. In such cases, it should be emphasized that confidentiality will nonetheless be preserved, for example through secure data handling and the exclusive reporting of results in aggregated form. Given that recruitment on LinkedIn and similar platforms can take place at very specific levels (e.g., by employer, job title, industry niche, or narrowly defined professional segments), the risk of de facto identifiability of respondents may be higher than in general-population web surveys. This applies in particular when small organizations or rare occupations are involved. Ethical scrutiny is therefore essential. Targeting strategies should avoid extremely small and potentially identifiable groups where this is not strictly necessary for the research aim, and analysis and reporting should follow minimum cell-size rules to prevent the indirect identification of individuals or specific organizations. It is strongly recommended to seek approval or at least formal advice from the relevant ethics committee at the home institution or from an external ethics body before fieldwork begins. An ethics review helps to ensure that the sampling and recruitment strategy is appropriate, that privacy and confidentiality risks are minimized, that the informed consent procedure is adequate, and that the study aligns with existing ethical guidelines and professional codes of conduct.

Application example

This section illustrates how the LinkedIn recruitment workflow described above can be applied in practice. It summarizes the design, implementation, and outcomes of an exemplary advertising campaign conducted as part of a master’s seminar at Bielefeld University in summer term 2024 titled “Sociological methods - quantitative: Survey-based measurement of inclusion and diversity in companies: Development and implementation of employee surveys”, taught by Zaza Zindel and Lisa de Vries.
Within this seminar, students developed a questionnaire to measure inclusion and diversity in workplace settings. The instrument comprised 37 items, including the following topics:

• Working life (7 items)
• Career orientation (3 items)
• Diversity (3 items)
• Discrimination (4 items)
• Health (6 items)
• Sociodemographic (13 items)
• Closing question (1 item).

The target population consisted of employees in Germany aged 18 years or older. The survey was programmed using the software LimeSurvey. To reach the target population without cooperation from specific companies, survey recruitment was carried out via LinkedIn ads. A total budget of 500 Euro was allocated for the campaign.

Ad campaign details

Following the three-tier campaign structure described above, the campaign was configured at the campaign, ad set, and ad levels. A dedicated LinkedIn page titled “Empirische Sozialforschung – Uni Bielefeld” (English translation: “Empirical Social Research – Bielefeld University” was created to host the ads and serve as the institutional identity for the recruitment process (see Figure 20).

Figure 20: LinkedIn page: Empirical social research – Bielefeld University

_{^{Source: LinkedIn Campaign Manager. Images by Monster Ztudio and Siberian Art via stock.adobe.com.}}

Ad creatives

Two separate advertisements in the single image format were developed. They used identical textual content but different ad images to examine how visuals influence engagement (Figures 21 and 22).

Headline: We want your opinion: Take part in our survey on the German labor market! Introductory text: Help us better understand the German labor market! We are looking for participants for a short survey that is part of a scientific study conducted by Bielefeld University. The survey is aimed at all employed people aged 18 and over in Germany.

Figure 21: Targeting ad 1 used to recruit participants on LinkedIn

_{^{Source: LinkedIn Campaign Manager. Image by mitay20 via stock.adobe.com.}}

Figure 22: Targeting ad 2 used to recruit participants on LinkedIn

_{^{Source: LinkedIn Campaign Manager. Image by Africa Studio via stock.adobe.com.}}

Campaign setup

Both advertisements employed identical targeting criteria to ensure comparability across creatives. To recruit broadly from employed individuals in Germany, targeting was restricted to:

• Location: Germany
• Profile Language: German
• Age: 18+ years

No additional targeting criteria were applied, and audience expansion was disabled. The placement was set to LinkedIn Feed only and audience network placements was disabled. A lifetime budget of 500 EUR was defined, optimization goal was landing page clicks, and the bidding strategy was maximum delivery until all the budget is spent.

Summary and key takeaways

LinkedIn provides a distinctive infrastructure for survey recruitment in professional and occupational populations. Its career-focused user base, comparatively low noise-to-signal ratio, and rich profile data on education, industry, and employment make it particularly suitable for studies on labor markets, workplace dynamics, and professional attitudes. In contrast to general-purpose platforms such as Instagram or TikTok, LinkedIn is designed around work, networking, and expertise. As a result, it tends to reach individuals of working age, often with medium to high levels of education and strong labor market attachment, which is reflected in the sample composition of the application example presented in this tutorial. This step-by-step guide provided an overview of how to use LinkedIn ads to recruit participants for survey research. The application example from Bielefeld University illustrates that with a moderate budget, LinkedIn can yield a substantial number of completed interviews at reasonable cost, while also highlighting typical patterns such as the overrepresentation of highly educated respondents and specific age groups.

Taken together, the key takeaways from this tutorial can be summarized as follows:

1. Alternative recruitment channel: LinkedIn offers a viable alternative to conventional sampling approaches and general social media platforms for reaching employed and occupationally defined populations.
2. Distinct user structure: Compared to platforms such as Instagram or TikTok, LinkedIn tends to reach individuals in mid-career and higher educational strata, which is advantageous for many labor market-related studies.
3. Fine-grained targeting: Detailed targeting by job title, industry, company size, skills, and other professional attributes enables access to niche populations that are otherwise difficult to sample.
4. Selective coverage and bias: Only certain groups are reachable via LinkedIn, and topic-specific visuals or messaging can reinforce self-selection, requiring careful interpretation of substantive findings.
5. Heightened ethical responsibilities: Because targeting can be highly specific, ethical issues around privacy, identifiability, and fair treatment are particularly salient and call for robust consent procedures, conservative reporting practices, and prior ethics review.
6. Potential stability over time: As a professional networking platform, LinkedIn may be less prone to rapid shifts in usage patterns than entertainment-oriented social networks, potentially providing a relatively stable environment for recurring or longitudinal recruitment; however, this requires continued empirical monitoring.
7. Need for further evidence: Future studies should systematically evaluate recruitment efficiency, sample quality, and bias across different designs and topics and explore, where ethically permissible, whether and how organizational or contextual information can be incorporated without compromising participant protection.

Further reading

Researchers interested in using social media platforms for survey recruitment may benefit from the following literature, which covers methodological, ethical, and practical aspects of recruiting survey respondents via social media advertisements. Together, these publications provide guidance on campaign design and performance, ethical and legal considerations, data quality, and the assessment of non-probability samples.

• Höhne, J. K., Claassen, J., Kühne, S., & Zindel, Z. (2025). Social media ads for survey recruitment: Performance, costs, user engagement. International Journal of Market Research. https://doi.org/10.1177/14707853251367805
• Pötzschke, S., Weiß, B., Daikeler, J., Silber, H., & Beuthner, C. (2023). A guideline on how to recruit respondents for online surveys using Facebook and Instagram: Using hard-to-reach health workers as an example. Mannheim: GESIS – Leibniz Institute for the Social Sciences (GESIS Survey Guidelines). https://doi.org/10.15465/gesis-sg_en_045
• Rohr, B., Felderer, B., Silber, H., Daikeler, J., Roßmann, J., & Schröder, J. (2024). When are non-probability surveys fit for my purpose? Mannheim: GESIS – Leibniz Institute for the Social Sciences (GESIS Survey Guidelines). https://doi.org/10.15465/gesis-sg_en_050
• Zimmermann, B. M., Willem, T., Bredthauer, C. J., & Buyx, A. (2022). Ethical issues in social media recruitment for clinical studies: Ethical analysis and framework. Journal of Medical Internet Research, 24(5), e31231. https://doi.org/10.2196/31231
• Zindel, Z., Kühne, S., Perrotta, D., & Zagheni, E. (2025). Ad images in social media survey recruitment: What they see is what we get. International Journal of Social Research Methodology, 1–20. https://doi.org/10.1080/13645579.2025.2597303
• Zindel, Z. (2026). Should we worry about problematic response behaviour in social media surveys? Understanding the impact of social group cues in recruitment. Survey Methods: Insights from the Field. https://doi.org/10.13094/SMIF-2026-00015
• Zindel, Z. (2023). Social media recruitment in online survey research: A systematic literature review. Methods, Data, Analyses, 17(2), 207–248. https://doi.org/10.12758/mda.2022.15

About the authors

Dr. Zaza Zindel is a research associate at the German Centre for Integration and Migration Research (DeZIM). She is a social scientist working on survey methodology, data science, and social media research to study vulnerable, rare, and underrepresented populations and their experiences of discrimination. Her work combines methodological innovation with substantive applications using survey data, experiments, and new data sources, drawing on computational and data science approaches.

Dr. Lisa de Vries is acting professor at FernUniversität in Hagen. Her research focuses on discrimination and inequality based on sexual orientation and gender from a quantitative perspective. Her research interests are sexual and gender minorities, labor market and educational inequalities, global developments of social acceptance and equality, and the inclusion of minority populations in research and surveys.

Overview

1. Introduction to social media survey recruitment
2. Recruiting via TikTok – a step-by-step guide
   1. The TikTok Ads Manager
   2. Setting up an ad campaign
      1. Campaign level
      2. Ad group level
      3. Ad level
3. Application example
   1. Ad campaign details
   2. Campaign results
   3. Sample composition
4. Ethical considerations
5. Summary and key takeaways
6. Further reading

Introduction to social media survey recruitment

Recruiting research participants is a crucial component of any survey study. However, traditional recruitment methods, such as random walk procedures or random digit dialing, can be time-consuming and expensive. With the growing popularity of social media, platforms like TikTok provide a unique opportunity to quickly and effectively recruit a diverse range of survey participants. TikTok is a popular social media app that allows users to create and share short-form videos with a global audience. With a self-reported number of over 1 billion active users, the platform has gained immense popularity, particularly among younger generations (TikTok, n.d.). TikTok is relevant and interesting for social science research due to its unique format and engaging nature. The app provides a window into the cultural and social trends of different communities, making it an attractive platform for studying human behavior and attitudes. Additionally, TikTok’s algorithm allows for a personalized feed, meaning that users are served content based on their interests and behaviors. This feature makes it an interesting platform for reaching and studying specific populations.

As a potential tool for survey recruitment, TikTok ads offer several advantages. The platform’s vast and diverse user base provides researchers with access to a broad range of potential participants, while its targeting options allow them to reach very specific populations that may be difficult to reach through more traditional recruitment channels. Unlike social media platforms like Facebook and Instagram, TikTok’s structure revolves around short-form videos set to music, offering a unique and visually engaging format that researchers can utilize to capture users’ attention and encourage their participation. By leveraging visually appealing videos, researchers can create compelling survey invitations, enhancing the overall survey experience. Furthermore, TikTok’s predominantly young user base sets it apart from other platforms, making it particularly valuable for studies targeting younger demographics. This demographic distinction allows researchers to gain insights from a generation with unique attitudes, behaviors, and perspectives, ultimately contributing to a more comprehensive understanding of societal trends and dynamics.

Recruiting via TikTok – a step-by-step guide

In the following section, we will walk you through the step-by-step process of setting up advertisements on TikTok. We will highlight the important factors you need to consider in order to generate a diverse and comprehensive sample for a web survey.

The TikTok Ads Manager

Having an active TikTok account is the first step towards advertising and recruiting survey participants on TikTok. This is because only individuals and organizations with registered business accounts are allowed to run ads on the platform. To register for a business account, go to the TikTok for Business website and proceed by clicking the “Create Now” button (see Figure 1).

Figure 1: Starting page of TikTok for Business website.

_{^{Source: Tik Tok for Business}}

From there, follow the prompts to sign up for a new account. You will need to provide basic information such as your email address and a password to create an account. Once you have an account, you can proceed to set up your business profile (see Figure 2). This includes providing information such as your company name, industry and country. Make sure to accurately and clearly represent your business, as this will help to build trust with potential survey participants.

Figure 2: TikTok for Business profile.

_{^{Source: Tik Tok for Business (after login)}}

Setting up an ad campaign

When creating an advertising campaign on TikTok, it is important to familiarize yourself with the campaign structure. The campaign settings are structured into three distinctive hierarchical levels: campaign level, ad group level, and ad level (see Figure 3). Within the ad campaign, you create ad groups, which serve as subdivisions allowing for further targeting refinement. Each ad group can contain multiple individual ads that serve as the creative content displayed to TikTok users. This means that within a single ad campaign, you can create multiple ad groups, and within each ad group, there can be multiple ads. Each level requires making informed choices and providing necessary information to ensure the effectiveness of your ad campaign.

Figure 3: TikTok ad hierarchy.

Campaign level

At the campaign level, the first step is to choose the campaign objective that best aligns with your advertising goal. This decision will determine how TikTok will optimize the ad delivery. TikTok offers three main categories of campaign objectives: awareness, consideration, and conversion (see Figure 4). Each category serves a distinct purpose in terms of user interaction with your ad and the desired actions you want users to take.

Figure 4: Selecting an advertising objective.

_{^{Source: Tik Tok Ads Manager (after login)}}

Firstly, awareness campaigns are aimed at sparking interest and enhancing the visibility of a brand, product, or service. These campaigns are designed to reach as many people as possible and create a lasting impression of a brand. Secondly, consideration campaigns are focused on driving engagement and interest. This includes encouraging users to visit a website, interact with content, or follow an account. These campaigns are designed to provide users with more information about a brand, product, or service, and to motivate them to take a specific desired action. Lastly, conversion campaigns are designed to drive users towards a specific action, such as making a purchase or downloading an app.

When creating a TikTok ad campaign to recruit survey participants, we highly recommend setting the campaign objective to “Traffic”, which is categorized under consideration. This objective focuses on driving traffic to a designated landing page, such as your survey website. By selecting this objective, your ad campaign will be optimized to effectively direct users to your survey website, increasing the likelihood of attracting potential participants to take part in your survey. TikTok’s machine learning algorithms will be leveraged to identify users within the target audience who are most likely to take the desired action of clicking on the ad and visiting your landing page.

After selecting the campaign objective, you can proceed to define some essential settings of your campaign. These settings include providing a campaign name and defining a maximum budget for the entire campaign (see Figure 5). While the maximum budget is not mandatory, we recommend setting it as a precautionary measure to prevent your campaign from exceeding the spending limit of your research project.

Figure 5: Defining campaign name and campaign budget.

_{^{Source: Tik Tok Ads Manager (after login)}}

Ad group level

At this level, you can create multiple ad groups within a campaign, each with its own settings tailored to the target audience, thereby enhancing ad effectiveness. You can specify various settings unique to each ad group. These include the optimization location, placements, targeting, budget and schedule, bidding, and optimization. First, set the optimization location, i.e. where to direct traffic. For survey recruitment, the optimization location should be set to the survey website where participants can complete the survey. Under the traffic objective, ad placement in Germany is limited to TikTok and Pangle (see Figure 6). While TikTok is the primary platform where ads will appear, Pangle is an ad network that provides access to third-party apps. We recommend to choose TikTok instead of Pangle, as TikTok has a larger user base and provides more advanced targeting options, which can help you reach a more specific audience. For other countries, other ad placements are available (for an overview see TikTok (n.d.)). Additionally, within the placements section, you have the option to customize your ad settings to control the level of user engagement. Consider deactivating comments to avoid harmful or violent reactions, disabling video downloads to protect your content, and restricting video sharing to ensure that your ads are exclusively viewed within the TikTok platform.

Figure 6: Defining placement options.

_{^{Source: Tik Tok Ads Manager (after login)}}

Targeting is another crucial step at the ad group level where you define the audience that your ad will be shown to. TikTok provides a range of targeting options, including demographic information, interests, behaviors, and custom audiences. You should define the target audience based on your research objectives and select the appropriate targeting options to effectively reach your desired audience. Take advantage of the wide range of targeting options available when running TikTok ads. Here are some examples:

• Demographic targeting: Gender (binary), age groups, and language preferences (see Figure 7).
• Geographic targeting: Countries/regions you can target using “Location” depending on the country/region associated with your TikTok For Business account. For a comprehensive list of all available locations per country, see TikTok (n.d.).
• Interest-based targeting: Reach users based on their inferred interests, derived from their activity on TikTok (see Figure 8).
• Behavioral targeting: Reach users based on their behavior on TikTok, such as the videos they have liked, shared, or commented on (see Figure 8).

Figure 7: Defining demographic targeting parameters.

_{^{Source: Tik Tok Ads Manager (after login)}}

Figure 8: Defining interest- and behavior-based targeting.

_{^{Source: Tik Tok Ads Manager (after login)}}

It is crucial to ensure compliance with TikTok’s advertising policies and avoid any form of discriminatory targeting towards protected groups. TikTok maintains strict advertising policies to uphold community standards and legal requirements. Advertisers are expected to follow these policies, which are regularly updated, in order to maintain the quality and integrity of the TikTok platform. For a detailed description of all prohibited and restricted content, please refer to TikTok (n.d.).

The next step at the ad group level is to set a budget and schedule. You should define a daily budget for each ad group and specify the display schedule for each group. TikTok offers an estimation of an ad group’s potential reach based on the selected targeting options. This estimation can assist in determining an appropriate budget for your campaign (see Figure 9).

Figure 9: Estimated audience reach.

_{^{Source: Tik Tok Ads Manager (after login)}}

Once you have set the budget and schedule for your ad group, the next step is to define the options in Bidding & Optimization (see Figure 10). This step determines how the ad will be delivered and optimized for maximum effectiveness. On TikTok, you have the choice between two optimization goals: Click and landing page view. The default and recommended optimization goal is Click, which focuses on driving users to click on the ad and visit the website. Alternatively, the landing page view optimization goal targets users who not only click on the ad but also waits for the landing page to load completely. This is particularly useful for campaigns that require users engagement with the content on the landing page.

Figure 10: Defining bidding and optimization options.

_{^{Source: Tik Tok Ads Manager (after login)}}

Furthermore, you also have the option to select between two bid strategies: lowest cost and bid cap. The lowest cost bidding strategy is a spend-based approach, where TikTok’s algorithm automatically adjusts the bid to generate as many results as possible at the lowest cost per result. This strategy is ideal for most campaigns. However, if you want to have more control over the budget allocation, you can choose the bid cap option. With bid cap, you can set a maximum bid amount for each link click, ensuring that the cost per result remains below your predetermined limit. This strategy can be particularly beneficial for campaigns with limited budgets or when you want to ensure that you are not overpaying for each link click.

Ad level

On the ad level, you can create a captivating advertisement that will be displayed to the target audience. The goal is to make the ad highly appealing and drive engagement among your target population. To achieve this, your ad should be eye-catching, and effectively convey the purpose of the survey. Given that TikTok is a platform centered around videos, we highly recommend incorporating a short and captivating video into your ad. The video should grab viewers’ attention and motivate them to take action. While the ad format can also be a single image or image carousel, it is advisable to include at least some dynamic elements to align with the platform’s style. Moreover, you must keep in mind that TikTok ads are primarily optimized for mobile devices. Hence, the ad should be visually appealing and easily viewable on smartphones, ensuring a seamless user experience.

Figure 11: Example ad design.

_{^{Source: Image by Macrovector via stock.adobe.com}}

In addition, you have the option to choose from various call-to-action buttons, such as “Learn More” or “Sign Up”, which are designed to encourage users to take action. When users click on the button, they will be directed to the survey (see Figure 11). Furthermore, it is important to ensure that the ad text and description convey a clear and concise message that motivates users to participate in the survey.

Once the ad campaign is created, it can be launched. However, it is important to note that TikTok will review the ads to ensure compliance with their guidelines and policies. It is essential to avoid displaying prohibited products or services, such as live animals, adult sexual products, casinos, military gear, or political candidates or parties, in the ad creatives or on the landing pages. A comprehensive list of prohibited content can be found at TikTok (n.d.).

Once the ads are live, you can utilize TikTok’s ad manager tool to track their performance. Monitoring the number of views, clicks, and conversions is essential to assess the reach, engagement, and overall effectiveness of the ads. This valuable information can be used to make necessary adjustments to the ad settings and enhance the performance of future ads.

Application example

The following section presents the results of an exemplary TikTok advertising campaign conducted as part of a master’s seminar at Bielefeld University. In this seminar, students were tasked with designing a questionnaire aligned with their individual research interests. The main topics were attitudes toward refugees and youth crime in the UK and Germany. To efficiently and cost-effectively generate a substantial sample size, the study was promoted through TikTok ads.

Campaign results

Table 2 provides an overview of the ad campaign’s performance metrics. Over the course of the 4-day campaign, it reached a total of 117,626 TikTok users. The videos or animated images were played 189,100 times, with the ads being displayed 205,071 times, as reported by the TikTok Ad Manager. The campaign incurred a total cost of 387.48 EUR and generated 3,603 clicks on the call-to-action button, representing a click-through rate of 1.76 percent. The average cost per click was 0.11 EUR, which is comparable to recruitment via other social media platforms like Facebook or Instagram (cf. e.g., Zindel 2022).

Analyzing the ad performance reveals variation depending on the target groups. Ads placed in Germany (ad groups: D1 & D2) reached a higher overall number of impressions compared to the UK. This discrepancy can be attributed to the less developed TikTok ads market in Germany compared to the UK, resulting in lower costs for link clicks due to reduced ad competition. Consequently, the same ad budget resulted in varying levels of impressions. Figure 12 illustrates the cost per click (CPC) by ad group. Initially, during the learning phase of the advertising algorithm, ads in the UK were slightly more expensive than those in Germany. Moreover, the ad group targeting English speakers in Germany (ad group: DE2) exhibited the highest cost per click over the short duration of the campaign. This can be attributed to the relatively lower proportion of TikTok users in Germany who use the platform in English compared to those who use it in German. Targeting a smaller user base incurs higher costs to reach a significant number of users, as reflected in the number of impressions in Table 2.

Figure 12: Average cost per click (CPC) for each ad group.

Examining the metrics distribution in Table 2 highlights significant variations in ad performance. Some ads were displayed and viewed more frequently than others within the same ad group. For instance, in ad group DE2, the ad YD2DE2, one of two ads that focused on the topic of youth delinquency (“YD” for short), had the highest allocation rate determined by the advertising algorithm, resulting in the highest number of clicks on the call-to-action button within this group. When considering the click-through-rate (CTR) for all ads, which represents the conversion rate from impressions to clicks, ads served more frequently tended to have comparatively higher CTRs. However, it is important to note that these ads may not necessarily have the highest CTR within their respective ad groups. For example, in the UK ad group, the ad RF2UK had a CTR of 2.66 percent, surpassing the most frequently displayed ad YD2UK with a CTR of 2.22 percent. This discrepancy arises from the fact that ads that initially garner significant attention in the initial learning phase of the advertising algorithm may not necessarily sustain the highest level of interest from the target audience throughout the entire ad runtime TikTok (n.d.).

Table 2: Results of ad performance.

Table 3 presents the conversion of link clicks per ad into started and completed interviews, along with the corresponding total cost and cost per completed interview. Out of the 3,603 link clicks, 1,846 (51.29 percent) resulted in started interviews, and a total of 500 interviews (13.88 percent) were completed. With a total campaign cost of 387.48 EUR, the average cost per completed interview was 0.77 EUR. Notably, variations can be observed across the ad groups. The ad group targeting German-speaking individuals in Germany (DE1) received the highest number of link clicks (n=1,676), resulting in the highest number of started interviews (n=983; 58.65 percent), as well as completed interviews (n=265; 15.81 percent). This specific target group was also the most cost-effective to reach, with an average cost of 0.49 EUR per completed interview. In contrast, the average cost per completed interview was 0.99 EUR for the DE2 ad group and 1.23 EUR for the UK group. This disparity highlights the differences in costs discussed earlier, which are influenced by the bidding system and the ad algorithm.

Table 3: Results of survey performance.

Sample composition

The final sample comprises three subsamples derived from the three different ad groups: English-speaking individuals in the UK (ad group: UK), German-speaking individuals in Germany (ad group: DE1), and English-speaking individuals in Germany (ad group: DE2). Group DE1 accounts for 265 completed interviews, representing 53.00 percent of all interviews. Group DE2 includes 130 interviews (26.00 percent) and group UK has 105 interviews (21.00 percent). The average age of the respondents is 37.22 years (see Figure 13), which is higher than expected considering TikTok’s popularity among younger generations. However, it is important to note that the average age in our sample still falls below the overall average age of 44.7 in the German population (Statistische Ämter des Bundes und der Länder, 2022) and the median age for the UK (UN DESA, 2022).

Figure 13: Age distribution in overall sample.

The average age within the DE1 ad group is 47.74 years, whereas it is 23.99 years for the DE2 ad group and 25.53 years for individuals in the UK. This indicates a strong selection bias towards older individuals in the German-speaking group, while the other ad groups exhibit a strong bias towards younger participants. Table 4 compared the age distribution of each ad group with the ad allocation information provided by TikTok. These selection effects are primarily influenced by the ad algorithm. Therefore, for studies aiming to achieve a demographically balanced age distribution, it is recommended to further stratify ad groups and run specific ads targeted at different age groups to mitigate similar age biases.

Table 4: Age group distribution by ad groups.

Regarding gender distribution, a larger proportion (61.52 percent) of male participants was identified in the sample, while only 27.05 percent identified as female (see Table 5). Additionally, 60 individuals (11.43 percent) in the sample reported identifying with a different gender. A comparison of the sample data with the information from the TikTok Ads-Manager reveals that this effect can also be partially attributed to the allocation of the ads by the advertising algorithm. The DE1 ad group was predominantly delivered to men, while the DE2 and UK ad groups did not exhibit such a strong gender effect in their ad display. It appears that a large proportion of respondents, assumed to be female, chose the third response option, “another gender”. However, it is important to note that we lack individual gender information within the TikTok platform.

Table 5: Gender distribution by ad groups.

Ethical considerations

When considering the use of TikTok for survey recruitment and advertisements in general, several ethical implications merit careful consideration. TikTok, a Chinese company owned by ByteDance with established ties to the Chinese state, raises pertinent concerns. Purchasing advertising space on this platform contributes to TikTok’s financial revenue, strengthening its market position as one of the fastest-growing social media platforms. Due to its indirect connection to the Chinese state, numerous government agencies, such as the European Commission and the European Council, have banned TikTok from their official devices. Moreover, some countries have opted for a complete prohibition of TikTok due to concerns encompassing cyber security, privacy, and misinformation.

With the approach outlined in this blog post, TikTok does not have access to the survey data at any time. The social media platform remains incapable of identifying which users have participated in our survey. To ensure that the survey data remains inaccessible to third parties at all times, we strongly advise against the utilization of tracking tools provided by TikTok, as well as other platforms.

Nonetheless, it is worth noting that TikTok creates interest profiles of users based on their interaction with advertised content – a practice also common among other social media platforms. However, considering TikTok’s indirect link to the Chinese state, the possibility that these interest profiles could be accessed by Chinese authorities cannot be disregarded. This potential vulnerability is especially concerning for marginalized population groups, such as LGBTQI* individuals. While TikTok may lack specific information regarding the gender or sexual identity of its users, it can infer from users’ interactions with content related to LGBTQI* topics that these users at least hold an interest in such matters.

For all researchers intending to establish financial collaborations with TikTok –- whether to recruit survey participants or promote other content –- we strongly recommend a preliminary consultation with their respective institutions, employers, and ethics committees. This step will help determine the feasibility of such projects within the ethical framework.

Summary and key takeaways

Recruiting survey participants through TikTok ads offers several advantages for researchers. TikTok’s vast and diverse user base allows access to a broad range of potential participants, and its targeting options enable reaching specific populations that may be challenging to reach through traditional methods. By following our presented step-by-step guide for setting up advertisements on TikTok, you can effectively generate a diverse and inexpensive sample for your web surveys. The example of a TikTok advertising campaign demonstrates the platform’s potential in efficiently and cost-effectively recruiting survey participants. We delved into its unique features, demographics, and engagement levels, considering its suitability for reaching a diverse audience. Additionally, we examined the advantages and disadvantages of utilizing TikTok, highlighting its ability to generate rapid and widespread exposure while also acknowledging the challenges it presents, such as limited insights in the algorithmic ad allocation and potential for high selectivity.

Overall, the key takeaways from this tutorial are as follows:

• TikTok offers a vast user base with diverse demographics, making it a promising platform for survey recruitment efforts aimed at reaching a wide audience.
• Ad campaigns on TikTok are structured into campaign levels, ad group levels, and ad levels, each requiring informed choices and necessary information for effectiveness.
• TikTok provides various targeting options, including demographics, interests, behaviors, and custom audiences, to reach the desired audience.
• Ad creatives on TikTok should be eye-catching, incorporate videos or dynamic elements, and have clear ad text and call-to-action buttons.
• It is essential to carefully monitor and address potential ethical concerns, such as data privacy and user consent, when utilizing TikTok for survey recruitment.
• Monitoring the performance of TikTok ads through the ad manager tool is crucial for assessing reach, engagement, and effectiveness, especially when it becomes apparent that certain ad groups outperform others.
• The distribution of ads by the algorithm may lead to overrepresentation of different subgroups. For demographically more comparable population distributions, it is recommended to stratify the ads using the targeting parameters.

Further reading

Here are several publications that offer a valuable introduction to the topic of social media ads for survey respondent recruitment.These publications provide valuable insights into the use of social media ads for recruiting survey respondents and serve as a great starting point for exploring this subject further:

• Iannelli, L., Giglietto, F., Rossi, L., & Zurovac, E. (2020). Facebook Digital Traces for Survey Research: Assessing the Efficiency and Effectiveness of a Facebook Ad–Based Procedure for Recruiting Online Survey Respondents in Niche and Difficult-to-Reach Populations. Social Science Computer Review, 38(4), 462–476. doi:https://doi.org/10.1177/0894439318816638
• Kühne, S., & Zindel, Z. (2020). Using Facebook and Instagram to Recruit Web Survey Participants: A Step-by-Step Guide and Application. Survey Methods: Insights from the Field. doi:https://doi.org/10.13094/SMIF-2020-00017
• Neundorf, A., & Öztürk, A. (2021, July 26). How to Improve Representativeness and Cost-effectiveness in Samples Recruited through Meta: A Comparison of Advertisement Tools. url:https://doi.org/10.31219/osf.io/3g74n
• Neundorf, A., & Öztürk, A. (2021, December 9). Recruiting Research Participants through Facebook Advertisements: A Handbook. url:https://doi.org/10.31219/osf.io/87rg3
• Zindel, Z. (2022). Social Media Recruitment in Online Survey Research: A Systematic Literature Review. methods, data, analyses, 0, 42. doi:https://doi.org/10.12758/mda.2022.15

We would also like to highlight the excellent webinar series of the DEMED project. This series offers a range of insightful presentations that delve into the advantages of social media for survey research.

About the authors

Zaza Zindel is a research assistant and a doctoral researcher in sociology at Bielefeld University (Germany) with a specialization in survey research. Her dissertation centers around the utilization of social media as a means to recruit rare populations for web surveys. Her research interests encompass survey methodology, the potential of social media for empirical social research, and the exploration of new technologies to improve statistical representation of marginalized, vulnerable, or rare population groups.

Simon Lütkewitte is a research assistant at Bielefeld University (Germany) specializing in empirical social science research, particularly quantitative methods . He is also a PhD student at the Bielefeld Graduate School for History and Sociology (BGHS). His research focuses on values and attitudes within the sphere of sports. Methodologically, he is interested in applying statistical methods such as multilevel modeling, matching techniques, and social network analysis. In his dissertation project, he utilizes quantitative methods to examine the relationship between sports participation and gender-related attitudes and beliefs.

Overview

1. Introduction to social media & Twitter API v2
2. Academic research track application process
   1. Prerequisites
   2. Application
   3. After the application
3. Using the API
   1. Postman-as-a-playground
   2. Which package should I choose?
   3. academictwitteR: a code walkthrough using R
4. Preparing for methods: working with textual data
5. Reproducibility of research based on Twitter data
6. Further readings

Introduction to social media and Twitter API v2

Social media posts are full of potential for data mining and analysis. Despite problems tackling fake accounts and bots on the platform, it can be a very fruitful source to tackle research questions in a bandwidth of disciplines, including social sciences (e.g., Barberá 2015; Nguyen et al. 2021; Valle-Cruz et al. 2022; Sältzer 2022). Recognizing this potential also for commercial usage, platform providers increasingly restrict free access to such data.

Especially Twitter is an important data source with its richness of social and political interactions. As well Twitter did not offer a free-of-charge option to implement a full archive search of all tweets and users. Back then, the free version of API v1.1 was very limited with a maximum of 3,200 tweets or the past seven days of tweets. In addition, the range of available meta data¹ as well as implemented query² options were rather small. These limitations were lifted by the introduction of the redeveloped and rearranged Twitter API v2 in January 2021.³ For academic purposes, they opened up access to all available tweets and other objects posted on Twitter without any monetary costs for the researcher.

While this blog post focuses on the retrieval of textual data, Twitter content certainly offers more. Looking at social network interactions (e.g., followers, likes, …) is just one of the opportunities beyond text to reveal valuable information. This can be, for example, the usage of follower networks to estimate ideological positions (e.g., Barberá 2015) or measuring the importance of a user in a social network based on social interaction data.

Academic research track application process

As Application Programming Interfaces (APIs) are powerful tools which allow access to vast databases full of information, companies offering them are increasingly careful about who is allowed to use them. While the previous version of the Twitter API provided access without a dedicated application (for a detailed description, see this Methods Bites tutorial), the novel version requires you to go through an application process where you have to provide several details about you and your project with Twitter. This information includes data regarding yourself as well as the research project where you intend to work with Twitter data.

Application

The whole process can be initiated by clicking Apply on the official Twitter API academic research track Website. You’ll be asked to log in with your personal Twitter account.

Figure 1: Twitter Application Steps for academic research track API access.

_{^{Source: Twitter API Application Process}}

The figure above visualizes the steps you have to complete before your application can finally be submitted for Twitter’s internal review:

1. Basic Info: such as phone number verification and country selection
2. Academic Profile: such as link to an official profile (department website or similar) and academic role
3. Project Details: such as information about findings, description of the project itself, and how the API should be used there (e.g. methodologies and how the outcomes will be shared)
4. Review: provides an overview of the previous steps
5. Terms: developer agreement and policy

Before starting, it is recommended to carefully read which kind of career levels, projects and data behaviors are not allowed to use the API and thus have a high chance of receiving a refusal for their application. To give an example, if you plan to share the content of tweets publicly, you most probably won’t get access to the API as this would violate the Twitter rules. Again, more detailed information about this can be found on the Twitter API academic research track and Developer Terms information guides.

Step one requests generic information about your Twitter account while in step two you have to provide information about your academic profile. This includes a link to a publicly available record on an official department website and information regarding the academic institution you are working in. The third step is the most sophisticated one: your research project. It asks for short paragraphs about the project in general, what and how Twitter data is used there, and how the outcome of your work is shared with the public. The last two steps, review and terms, do not require any user-specific input but provide an overview of all filled-in information as well as the chance to read the developer agreement and policy.

Using the API

After your successful application, the Twitter Developer Portal is there to manage projects and environments (which belong to a project), generate API keys (“credentials” for API access), get an overview of real-time monthly tweet cap usage, check available API endpoints and their specifics and more.

After the creation of a project, an environment can be added and API keys generated.

Figure 2: API keys of an environment

The following keys are generated automatically and used depending on the API interface (e.g. the R package) at hand:

• API key \(\approx\) username (also called consumer key)
• API key secret \(\approx\) password (also called consumer secret)
• Bearer token \(\approx\) special access token (also called an authentication token)

It is crucial to keep them private and not push them to GitHub or similar! Otherwise someone else could gain access to your API account. Instead, store them somewhere locally or directly within an environment variable. The package we’re going into detail later on this blog post is guiding you safely through this process.

However, in case you’re plan to use them in other applications, you can store your keys in different ways. The most common way in R is to add them to the .Renviron file. To do this with comfort, install the R package usethis and call its method usethis::edit_r_environ() which lets you edit the .Renviron in the home directory of your computer. In the following you can add tokens (or anything else you want to keep stored locally) using this format:

Key1=value1
Key2=value2
# ...

After saving the file you can access values by calling Sys.getenv("Key1") within your R application. More best practices on managing your secrets can be found on the website APIs for Social Scientists.

## Save package names as a vector of strings
pkgs <- c("dplyr", "academictwitteR", "quanteda", "purrr")

## Install uninstalled packages
lapply(pkgs[!(pkgs %in% installed.packages())], install.packages)

## Load all packages to library and adjust options
lapply(pkgs, library, character.only = TRUE)

academictwitteR::set_bearer()

## Instructions:
## ℹ 1. Add line: TWITTER_BEARER=YOURTOKENHERE to .Renviron 
##      on new line, replacing YOURTOKENHERE with  actual bearer token
## ℹ 2. Restart R

german_mps <- read.csv("data/MP_de_twitter_uid.csv",
                       colClasses=c("user_id"="character"))
head(german_mps)

##              user_id                   name party
## 1           44608858       Marc Henrichmann   CDU
## 2 819914159915667456      Stephan Pilsinger   CSU
## 3         1391875208   Markus Alexander Uhl   CDU
## 4          569832889 Sigmar Hartmut Gabriel   SPD
## ...

(putin OR selenskyj) -is:retweet lang:en has:geo

# function to retrieve tweets in a specific time period of a single user
# (list of user IDs would be possible but one should keep
# the max. query string of 1024 characters in mind)

get_tweets_from_user <- function(user_id) {
  # Another option is to add "query" parameter
  academictwitteR::get_all_tweets(
    users = user_id,
    start_tweets = "2021-01-01T00:00:00Z",
    end_tweets = "2021-09-30T00:00:00Z",
    data_path = "data/raw/",
    n = 100)
}

purrr::walk(german_mps[["user_id"]], get_tweets_from_user)

# concatenate all retrieved tweets into one dataframe and select which columns
# should be kept
# Another option: set parameter "user" to TRUE to retrieve user information
tweets_df <- academictwitteR::bind_tweets(data_path = "data/raw/",
                                          output_format = "tidy") %>%
  dplyr::select(
    tweet_id,
    text,
    author_id,
    user_username,
    created_at,
    sourcetweet_type,
    sourcetweet_text,
    lang
  )

write.csv(tweets_df, "data/raw/tweets_german_mp.csv", row.names = FALSE)

Preparing for methods: working with textual data

You are now ready to move on! The usual steps applied to textual data (lowercasing, stopwords removal, stemming, …) depending on the method at hand can be used for pre-processing your tweets. Additional fine-tuning of these steps could involve the removal of, e.g. party IDs, URLs, user mentions or similar. Such steps can be easily done by using regular expressions (regex). Regular expressions are used to extract patterns in texts which then can be used for further analysis, removal or replacement. Many tutorials available on the web (e.g. RegexOne interactive tutorial) make it straightforward to learn how to bring such expressions into action within your domain.

The following code provides a first starting point for applying pre-processing steps. The code relies on the package quanteda which is an R package that is often used when working with text data in R. If you want to dive deeper into text mining and text analysis, Methods Bites has more blog posts on these topics.

tweet_corpus <- quanteda::corpus(tweets_df[["text"]],
                                 docnames = tweets_df[["tweet_id"]])

The code first transforms the dataframe of tweets into another data format, corpus, keeping the tweet_id as an identifier attached to each tweet text. Having the tweets in the corpus format makes it easy to apply pre-processing steps after tokenizing. The following list shows a selection of common methods. However, it is important that the decision, on which methods are applied, heavily relies on the following text processing approach:

• remove_punct: removes all punctuation
• remove_numbers: removes all numbers
• dfm_tolower(): applies lowercasing
• dfm_remove(stopwords("german")): removes German stopwords which occur very frequently
• dfm_wordstem(language = "german"): applies German stemming (e.g., wurden \(\rightarrow\) wurd)

# "2020 wurden in Berlin ca. 18.800 Miet-
# in Eigentumswohnungen umgewandelt. #Umwandlungsverbot"
dfm <-
  quanteda::dfm(tweet_corpus %>%
                  quanteda::tokens(
                    remove_punct = TRUE,
                    remove_numbers = TRUE)) %>%
  quanteda::dfm_tolower() %>% # removes capitialization
  quanteda::dfm_remove(
    stopwords("german")) %>% # removes German stopwords
  quanteda::dfm_wordstem(
    language = "german") # transforms words to their German wordstems
# "wurd berlin ca miet- eigentumswohn umgewandelt #umwandlungsverbot"

The function dfm (called above) returns a sparse document-feature matrix which could be a fruitful starting point for first-word frequency analysis:

head(dfm)

## Document-feature matrix of: 6 documents, 87 features (79.77% sparse)
## and 0 docvars.
##                       features
## docs                  leb plotzlich mehr schablon gut bos pass 😉 #esk #miet
##   44608858            1   1         1    1        1   1   1    1  1    1
##   819914159915667456  0   0         0    0        0   0   0    0  0    0
##   1391875208          0   0         0    0        0   0   0    0  0    0
##   569832889           0   0         1    0        0   0   0    0  0    1
## ...

You are finally at the step of applying further methods to tackle your research question and getting deeper insights into your crawled tweets. There is much more to explore: You can find further text-as-data tutorials on our blog.

Reproducibility of research based on Twitter data

As reproducible results are one of the major requirements of research projects, it has to be discussed how this could affect your work with Twitter data. The Twitter development agreement includes a clear statement of what researchers are allowed to publish along with their work:

“Academic researchers are permitted to distribute an unlimited number of Tweet IDs and/or User IDs if they are doing so on behalf of an academic institution and for the sole purpose of non-commercial research. For example, you are permitted to share an unlimited number of Tweet IDs for the purpose of enabling peer review or validation of your research.”⁷

This means that the content of tweets must not be shared publicly. As tweets can be deleted or accounts can be suspended this certainly states an issue for subsequent researchers attempting to replicate the findings as they won’t be able to recrawl such tweets via the API. However, there are also platforms like polititweet.org, which track public figures and based on that justify the publication even of deleted tweets:

Figure 3: polititweet.org section of the landing page

To conclude, this makes the decision of how to share what kind of data not easier. But one still has to ensure to choose the best available option to share his or her data without violating the Twitter rules which are as of today to at least share tweet IDs amongst the community.

Conclusion

This blog post provides a first glimpse into the academic research track Twitter API and the information richness of Twitter data. As there certainly will be further updates and changes to the API in the future, there are plenty of easy-to-use packages that build on active user communities. The community is there to keep the packages updated accordingly to the current Twitter API version. While there exist a lot of powerful packages to tackle the data gathering step, researchers still need to think carefully about how to further process the crawled information depending on their research question and method as well as how to make their research accessible to the community in an open science approach.

Further readings

• Official Twitter API Documentation
• Shamanth Kumar, Fred Morstatter, and Huan Liu. 2013. Twitter Data Analytics. Springer Publishing Company, Incorporated.
• Denis Cohen and Simon Kühne. 2019. Collecting and Analyzing Twitter Data Using R. Methods Bites.

About the author

Andreas Küpfer is a graduate of the Mannheim Master in Data Science and a doctoral researcher at the Technical University of Darmstadt. His interdisciplinary research interests include text-as-data, applying machine learning technologies, and substantial inference in the fields of political communication and political competition.

Overview

1. Introduction
2. Research Design
   1. Sampling and Power
   2. Ethical Approval
3. Pre-Registration
4. Data Collection
   1. Online Access Panels
   2. Programming
   3. Pilot Testing and Data Collection
5. Data and Code Sharing
6. Conclusion

Introduction

In this blog post, I will walk you through the process of fielding your own survey experiment using an online access panel. Online access panels are pre-selected groups of internet users who are paid to participate in various surveys, often including market research as well as scientific studies. You may have heard of panel providers like Bilendi/Respondi, Dynata, Kantar, and YouGov that are frequently used in social science research. They have become popular among social scientists because they are a relatively fast, easy, and cheap way to reach a sample of the general population, but there are certainly pros and cons that you should consider, as I will describe in this post.

Throughout this post, I will use a recent data collection as an example, a survey experiment that I conducted with Johanna Gereke (whose expertise I rely on extensively for this piece) and several other colleagues about demographic threat and how it affects group boundaries in the German context (Gereke et al. 2022). In some ways, this project is unique: it was completed as part of a replication seminar at the University of Mannheim, in which we worked with 12 Master’s and PhD students who helped plan the survey, analyze data, and write the first draft of our paper. But in many other ways, our experiences should apply to a wide range of potential surveys, and I hope that you will find it useful no matter the size of your research team or the topic you plan to study.

So, what steps are required to plan and execute a survey experiment? Your experience may vary depending on the complexity of your planned survey as well as the population you plan to sample from, but you can get an overview of our recent experience in the timeline shown in the figure below. This project was completed relatively quickly, going from initial conception to journal submission in just over 6 months. This timeline was intentionally fast, as we wanted the students to experience the whole data collection process in a single semester, but this is also an example of just how fast you can collect data using an online access panel. As you can see, the most time-consuming part of our research was designing the survey, but also note that ethical approval can take some time (more on this later). While your workflow may look a bit different, I would generally recommend proceeding in a similar order. In particular, ethical approval must come early in the process as you need this to collect any data. Tackling elements like this (and also procurement) that are out of your control early on can save you from delays later.

Figure 1: Project timeline.

Research Design

For our seminar, we chose to first replicate and then extend Maria Abascal’s recent work on group boundaries in the U.S. context (Abascal 2020). She finds that when white Americans are confronted with information about demographic threat, i.e. that they will no longer make up a majority of the U.S. population, they are more exclusive about who they see as “White”. We tested whether the same effect could be found in Germany, randomly assigning each of our participants to see one of two demographic projections that either predicted that people with a migration background would make up an equal share of the population in the future (treatment), or that they would remain a minority (control). After seeing this, participants rated pictures of 18 people as having a migration background or not. We hypothesized that following Abascal, the demographic threat treatment would make people more likely to classify people who appear ethnically ambiguous as having a migration background.

Ethical Approval

Once you have generally decided on your research design and your survey population, it is a good idea to start your application for ethical approval.

Based on our experience, the application itself is generally not too time-consuming unless your design brings up many ethical issues, but the approval process can take several weeks or even months depending on your institution. It has historically taken us about 4-6 weeks to get approval at the University of Mannheim. You should definitely plan ahead, because it is critical that you have this approval before beginning the data collection. This is not only to ensure that you are acting within the ethical and legal guidelines of your institution and your field of research, but also because many journals require ethical approval for any work they publish.

When do you need to apply for ethical approval? Your institution’s rules may vary, but the University of Mannheim requires it for any research on humans that:

• involves any personal or personally identifiable data
• deceives subjects
• involves psychological or physical health risks
• triggers strong emotions or asks about traumatic experiences
• manipulates subjects’ self-image
• involves minors, or
• presents risks to human dignity, life, health, and peaceful coexistence.

In our case, the first two bullets applied. Any demographic information about a respondent counts as personal data, even if it could not be used to identify any specific person, so your research will almost certainly collect some personal data. Additionally, we employed deception by withholding the purpose of the study from respondents until the end of the survey, which we had to address by debriefing all participants after the survey was completed. If you are unsure whether your study requires ethical approval or not, it is a good idea to ask the staff person associated with your institution’s ethics committee. You should also look at their website, which might have additional information to help you plan your application. (Researchers in Mannheim can find an application checklist and other information here.)

Your application should explain the ethical issues that will arise as part of conducting your research. You should explain why these features (like deception or data collection) are necessary for your design, and explain how you are minimizing any potential harms associated with them. You will also want to include as much information about your survey as you can, possibly including any planned debriefing text as well as the invitation to the survey, and maybe even the entire questionnaire depending on the expectations of your institution.

Another area that you may need to address in your application is data protection. This is perhaps especially important to researchers working in the European Union due to the General Data Protection Regulation (GDPR) passed in 2016. I am not a lawyer and cannot provide a legal opinion on what regulations may apply to your research, but luckily the BERD @ NFDI in Mannheim has put together a useful tool called the Interactive Virtual Assistant that can give you more information on this topic (unfortunately only in German for now, but an English version is planned). In general, you should try to limit the amount of personal data you collect as much as possible, and make a plan to store this data securely. You will also want to avoid collecting any information that could be used to uniquely identify any of your respondents (or “personally identifiable information”, also called PII). Note that PII is not limited to names or addresses, but also includes combinations of variables that could identify a single person (such as a combination of ethnicity, age, and postal code). The ethics committee may be especially critical of designs that require the collection of such information.

Pre-Registration

Once you have received ethical approval and have firmly decided on your design, you should consider pre-registering your research. A pre-registration is simply a report of your hypotheses, data source, and planned research design that you write and upload to a repository before starting data collection. This is meant to prevent practices like selective reporting and p-hacking, and to disclose confirmatory and exploratory analyses, which makes your decision-making more transparent to those who read and evaluate your work (Nosek et al. 2018). This is certainly not a required step, but one that can help you plan and motivate your research, and that promotes principles of open science. One important thing to note is that pre-registering your design does not forbid you from performing exploratory analyses or even making design changes later on; it merely requires that you disclose that you have done so (Simmons, Nelson, and Simonsohn 2021).

If you would like to pre-register your study, you can find several useful templates on OSF. A pre-registration will only require you to briefly describe your hypotheses, design, and planned analysis, information that you may have already collected for your ethics application. Once you have filled out your template, you can upload your pre-registration to a repository like OSF or AsPredicted (among others), where it will be given a timestamp to verify when it was posted. Once posted, your pre-registration can be embargoed, making it invisible for several months while you complete your research. Note that you can also create an anonymized link to your pre-registration if you would like to include it in a manuscript submitted for peer review. If you are looking for an example, here is our pre-registration, although note that ours uses the “Replication Recipe” template designed specifically for replications.

Data Collection

Data and Code Sharing

Once you have collected your data and finished your analysis, it’s time to share your results with the world! Of course you will write up your results in a manuscript, but you should also consider sharing your data and code with other researchers. Like pre-registration, this is optional, and this may not be feasible if your analysis relies heavily on proprietary data, but data sharing is becoming increasingly common in social science research, and may even be requested by some journals. Sharing your data and code fosters transparency and reproducibility, but there are also more self-interested reasons to share: uploading your work can open new opportunities for collaboration and generate more citations for your work.

When you are preparing to share your data and code, it is important to ask yourself: could another researcher reproduce your results without any additional information? In your dataset, this means ensuring clear coding and labeling, perhaps in a separate codebook file. It also means compiling your questionnaires so that people can see the source of the data (and translating them, if necessary). Perhaps most importantly, make sure that you anonymize or delete any personally identifiable information – refer back to your ethics application and be sure that you are doing what you promised to do. If your analysis includes other data sources that you do not own, you should also make sure to exclude these from your dataset (while informing users about where they can be accessed, and providing code that allows users to merge external data to your own). You can find more information on data preparation from ICPSR and GESIS. In your code, make sure you include labels to describe what each section does and that you reference all required packages. You should also have multiple people test your code, ideally someone who was not involved in the project, to ensure that everything is clear and that it works (ideally across different systems) as intended.

Once your code and data are replication-ready, you can upload them to a repository of your choice. You want to choose a repository that can ensure long-term preservation and a persistent identifier (such as a URL or DOI) so that people can find your information. Also make sure that your data will be accessible to other researchers, and if needed that it can be accessed anonymously during the peer review process. Some commonly used (and free!) repositories in the social sciences include OSF, Harvard Dataverse, ICPSR, and GESIS’ SowiDataNet|datorium. You might also check with your university library, as some institutions may have their own repository (like Mannheim’s MADOC).

Conclusion

If you have made it through all of these steps, congratulations on finishing your first data collection! As you can see, a lot of planning goes into any survey experiment. Despite this, I think it is worth the trouble. Collecting your own data can be a creative, rewarding, and even fun process that allows you to explore entirely new scientific questions. I hope this post has encouraged you to try it out for yourself, and I wish you the best of luck with your research.

Further readings

• Mutz, D. C. (2011). Population-based survey experiments. Princeton University Press.
• Auspurg, K., & Hinz, T. (2014). Factorial survey experiments (Vol. 175). Sage Publications.
• Bansak, K., Hainmueller, J., Hopkins, D., & Yamamoto, T. (2021). Conjoint Survey Experiments. In J. Druckman & D. Green (Eds.), Advances in Experimental Political Science (pp. 19-41). Cambridge: Cambridge University Press.
• Salganik, M. J. (2019). Bit by bit: Social research in the digital age. Princeton University Press.
• Callegaro, M., Baker, R. P., Bethlehem, J., Göritz, A. S., Krosnick, J. A. & Lavrakas, P. J. (2014). Online Panel Research: A Data Quality Perspective. Wiley.

About the presenter

Joshua Hellyer is a doctoral researcher at the Mannheim Centre for European Social Research (MZES). His research focuses on discrimination against ethnic and sexual minorities, particularly in the housing and labor markets.

Contents

1. How to use web log and related data for social research
2. How to obtain web log and related data
3. How to handle “big data”
4. About the presenter
5. Further reading
6. References

How to use web log and related data for social research

Web logs (“browsing histories”) are records of app use and search queries. They are highly interesting data sources for research in the social sciences as they offer detailed insights into human behavior. However, only a few studies have used such data in the social sciences so far. Examples of such studies include Stephens-Davidowitz (2014), who studied racial animus in the 2012 U.S. presidential election, Peterson, Goel, and Iyengar (2018) and Flaxman, Goel, and Rao (2016), who analyzed filter bubbles, echo chambers and partisan polarization in the U.S. and Guess, Nyhan, and Reifler (2020) who studied the spread of fake news in the U.S. In the Netherlands, Möller et al. (2019) analyzed online news engagement based on three different modes of news use. In Israel, Dvir-Girsman (2017) documented that audience homophily is higher among individuals with more extreme ideology and that it is associated with ideological polarization and intolerance. Bach et al. (2019) showed for Germany that online and mobile device activities predict voting behavior and political preferences to a limited degree only. Chancellor and Counts (2018) show that internet search data can be used to estimate employment demand in the U.S.

In other disciplines, similar data have been used, for example, to predict influenza activity (Ginsberg et al. 2009, but see @lazer2014parable) and to show that users who search for relatively harmless symptoms easily end up searching for serious diseases (White and Horvitz 2009). Another study demonstrates how concerns about pregnancy and childbirth change over the course of pregnancy (Fourney, White, and Horvitz 2015). Furthermore, several papers show how a variety of user attributes such as socio-demographics can be inferred from web logs, search queries and app records (see Hinds and Joinson 2018 for a recent overview). However, many of those studies have been conducted by researchers from computer science. While they often rely on search query data obtained from search engine providers like Bing and Google, they typically focus more on the technical aspects and on the evaluation of the performance of the underlying algorithms. As social scientists, however, we often focus on the theory-driven development of models and the testing of hypotheses about human and societal behavior. Thus, this blog post will focus on the latter.

One challenge that researchers face when designing studies that rely on web logs, records of app use, and search queries is how to get access to such data. Several of the studies mentioned above use large amounts of search query data from search engines like Google or Bing (Stephens-Davidowitz 2014; Chancellor and Counts 2018; White and Horvitz 2009; Fourney, White, and Horvitz 2015). These data are, however, usually only available if one teams up with researchers from the respective companies. Another way to obtain data is through commercial providers who keep opt-in panels of users who occasionally answer survey questions in exchange for money. Researchers can pay those vendors in order to get access to their panels and ask participants survey questions. In addition, several of these providers also offer web log and mobile device use records from users who (in exchange for additional pay) agreed to having their online mobile activities monitored. In this blog post, we will mainly focus on this latter way of obtaining data. Before we talk about this topic in more detail, we will briefly summarize what we mean when we speak of web logs, records of app use, and search queries.

To get a better understanding of such data, the table below shows a collection of a few artificial web logs made up for this blog post. Typically, we observe a person identifier (first column) for the person whose records we observe. Second, we have a URL (Uniform Resource Locator) column which tells us which URL this person visited, when (column “Timestamp”) and for how long (“Duration of use”; here, in seconds). The most interesting information in this table is the URL column. Even without a detailed understanding of the specific form of a URL, we can easily see that, in the first row, Person 1 visited a web address that seems to inform her about the weather in Mannheim. In addition, we observe when she visited this address and how much time she spent there. The second row tells us that this person likely sent (or received) a message to (from) Peter Mustermann. We cannot, however, observe the content of the message (which we also should not, given obvious privacy reasons). The third row shows that Person 1 then visited the Facebook page of the CDU. The fourth row shows that she watched a video on YouTube. If we accessed the web address (or programmed a scraping tool), we could also learn what the video was about. From the visit in the fifth row, we learn that this person read an article about the state of the German economy on DIE ZEIT, a German newspaper.

With respect to Person 2, we can also observe what items they searched on Amazon (sixth row) or which tweet they saw (seventh row). We also learn that they searched for the voting advice application “Wahl-O-Mat” (ninth row). From the URL in row 9, we can extract the exact words a person entered into a search engine (the “search queries”). From row 10, we also know that Person 2 then actually used this tool. Thus, we already see that we can learn a lot about users’ behavior, their interests and preferences.

We observe similar records regarding app use on mobile devices, as shown in the next table. Here, we observe the name of an app instead of a URL. Unfortunately, we cannot observe what users do inside the apps they use. That is, we do not know which articles they read when they open a newspaper app or which videos they watch on YouTube or Netflix. Yet, if somebody frequently opens newspaper apps, we might infer that this person may be interested in politics. Moreover, users who use an app called “Period Tracker” are likely female and we may even learn when they have their period by observing temporal variations in usage of this app. Similarly, somebody who uses an app that informs them about prayer times is likely religious. These are just a few examples that show how observing users’ web logs or app use can reveal information that users may perceive as sensitive personal information (see Bach et al. 2019 for a study on this topic).

It is important to note that participants of commercial panels can usually pause data collection on their devices temporarily. This might happen if they do not feel comfortable having their activities recorded, e.g. when they do online banking or watch movies from illegal streams. However, our own data collections show large amounts of potentially sensitive information (such as adult content, gambling, and illegal streaming), which suggests that users do not make use of this possibility very often.

How to obtain web log and related data

As mentioned earlier, the easiest way to obtain web log and app use data is through commercial vendors who operate online access panels (“non-probability panels”). So far, there seem to be only a handful of providers, such as respondi AG (Germany, UK and France, for example), YouGov (UK and US, amongst others) and netquest (mostly Spain and Latin America). For an overview of providers, see for example, Wakoopa Hub. Since such data have not been available for long, we do not know much about the quality of such data (for an exception see Revilla, Ochoa, and Loewe 2017).. In addition to accessing web logs and app use data, researchers can collect survey data for the same individuals. That is, by asking users questions we can enrich the web log data with important information about users’ socio-demographic characteristics, their voting behavior, or their political preferences. This information is usually not directly observable from the web logs and app use records, but likely makes the data much more valuable for social scientific research purposes. However, we should always keep in mind that due to the opt-in nature of these access panels, we need to be careful when making statements about the representativeness of our findings. That is, because most of our statistical estimators (e.g., for variances) rely on true probability sampling, we need to adjust our models and estimators, e.g. by using weights (for more information on non-probability sampling and non-probability panels see, e.g., Mercer et al. 2017; Cornesse et al. 2020).

Another way to obtain data on users’ online activities is through Google Trends. Briefly speaking, this service allows the estimation of the popularity of search queries in Google Search across various regions and languages. Several R packages in R allow users to automatically extract data from Google Trends – see, for instance, gtrendsR. Further details on Google Trends can be found here. A major drawback of these data is that they do not come with additional information about users and only provide information at aggregate levels. That is, one can learn only about the popularity of a specific search query compared to the popularity of other search queries, which is arguably much less informative than individual-level web logs.

A third way to obtain web log, app, and search query data is through developing one’s own research app. A few projects using this approach were launched in recent years. One of the most prominent examples is the IAB-SMART project (for details, see Kreuter et al. 2019). Researchers of the Institute for Employment Research (IAB) in Nuremberg, Germany, developed an app that has since been downloaded by several hundred respondents of the IAB’s panel survey “Labour Market and Social Security”. The app collects information on respondents’ smartphone activities and can access even more information than those described above (including geolocation and address books). While this approach is likely the most elaborate, it is also the most difficult and most costly one to implement: Researchers need to program their own tools and recruit participants on their own or piggyback on an existing study.

How to handle “big data”

Finally, a note on useful tools and prerequisites for analyzing web logs and records of smartphone use. First of all, the amount of data can quickly exceed the computational power of a standard desktop computer. Four months of web log data used in Bach et al. (2019), for example, contained about 38 million observations. Working with data of this size, researchers may have to consider using remote computing services like Digital Ocean, Amazon AWS or Microsoft Azure, which offer computational resources for little money through virtual servers. Second, understanding URL contents by observing single URLs is straightforward. Analyzing thousands of URLs, however, requires text mining and natural language processing (NLP) techniques if one wants, for example, to select only those URLs that point to news articles. Moreover, in addition to analyzing the title of a news article (which can often be observed from the URL alone), one might also want to analyze the whole content of the article. In such cases, in addition to being able to automatically extract the topic of an article through NLP techniques, knowing how to scrape website contents will likely also be helpful. Some useful materials are linked below.

About the presenter

Ruben Bach is a postdoctoral researcher at the University of Mannheim, focusing on social science quantitative research methods. His interests include topics related to big data in the social sciences, machine learning, causal inference, and survey research.

Further reading

• Meyer, Cosima and Cornelius Puschmann. 2019. Advancing Text Mining with R and quanteda. Methods Bites: Blog of the MZES Social Science Data Lab. Blog Post Tutorial.
• Munzert, Simon. 2016. Three easy-to-learn tools to scrape data from the Web with R. MZES Social Science Data Lab. Workshop Materials.

Contents

1. Wikipedia for Political Research
2. Collecting and Analyzing Wikipedia Data
   1. Application 1: Using Pageviews to Measure Public Attention
   2. Application 2: Using Article Links to Create a Network Graph of German MPs
   3. Application 3: Using Clickstream Data to Analyze Referral Patterns
3. Collecting Data via Wikidata Queries
4. legislatoR
   1. Application 1: Social Media Adoption Rates
   2. Application 2: Public Attention to Members of the German Bundestag
5. Conclusion
6. About the Presenter
7. References

Wikipedia for Political Research

According to its website, “Wikipedia […] is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content”. As of July 2019, it comprises more than 48 million articles and is ranked sixth in the list of the most frequently visited websites.

Wikipedia harbors numerous types of data. These include both article contents as well as meta information such as pageviews, clickstreams, links and backlinks, or edits and revision histories. Additionally, Wikipedia’s sibling, the collaboratively edited document-oriented data base Wikidata, provides access to over 58 million data items (as of July 2019). Given the broad collection of articles on politicians and institutions from all over the world, Wikipedia offers tremendous potential for (comparative) political research.

In what follows, we will introduce the functionalities of various R packages, including WikipediR, WikidataR, and pageviews. In doing so, we will showcase how to connect to Wikipedia and Wikidata APIs, how to efficiently access and parse content, and how to process the retrieved data in order to address various questions of substantive interest. We will also provide an overview of the legislatoR package, a fully relational individual-level data package that comprises political, sociodemographic, and Wikipedia-related data on elected politicians from various consolidated democracies.

## Packages
pkgs <- c(
  "devtools",
  "ggnetwork",
  "igraph",
  "intergraph",
  "tidyverse",
  "rvest",
  "devtools",
  "magrittr",
  "plotly",
  "RColorBrewer",
  "colorspace",
  "lubridate",
  "networkD3",
  "pageviews",
  "readr",
  "wikipediatrend",
  "WikipediR",
  "WikidataR"
)

## Install uninstalled packages
lapply(pkgs[!(pkgs %in% installed.packages())], install.packages)

## Load all packages to library
lapply(pkgs, library, character.only = TRUE)

## legislatoR
devtools::install_github("saschagobel/legislatoR")
library(legislatoR)

Collecting and Analyzing Wikipedia Data

# get pageviews
trump_views <-
  article_pageviews(
    project = "en.wikipedia",
    article = "Donald Trump",
    user_type = "user",
    start = "2015070100",
    end = "2017050100"
  )
head(trump_views)

clinton_views <-
  article_pageviews(
    project = "en.wikipedia",
    article = "Hillary Clinton",
    user_type = "user",
    start = "2015070100",
    end = "2017050100"
  )

# Plot pageviews
plot(ymd(trump_views$date), trump_views$views, col = "red", type = "l", xlab="Time", ylab="Pageviews")
lines(ymd(clinton_views$date), clinton_views$views, col = "blue")
legend("topleft", legend=c("Trump","Clinton"), cex=.8,col=c("red","blue"), lty=1) 

## step 1: get info about legislators
dat <- semi_join(
  x = get_core(legislature = "deu"),
  y = filter(get_political(legislature = "deu"), session == 19),
  by = "pageid"
)

## step 2: get page links (max 500 links)
if (!file.exists("studying-politics-wikipedia/data/wikipediR/mdb_links_list.RData")) {
  links_list <- list()
  for (i in 1:nrow(dat)) {
    links <-
      page_links(
        "de",
        "wikipedia",
        page = dat$wikititle[i],
        clean_response = TRUE,
        limit = 500,
        namespaces = 0
      )
    links_list[[i]] <- lapply(links[[1]]$links, "[", 2) %>% unlist
  }
  save(links_list, file = "studying-politics-wikipedia/data/wikipediR/mdb_links_list.RData")
} else{
  load("studying-politics-wikipedia/data/wikipediR/mdb_links_list.RData")
}

## step 3: identify links between MPs
# loop preparation
connections <- data.frame(from = NULL, to = NULL)
# loop
for (i in seq_along(dat$wikititle)) {
  links_in_pslinks <-
    seq_along(dat$wikititle)[str_replace_all(dat$wikititle, "_", " ") %in%
                               links_list[[i]]]
  links_in_pslinks <- links_in_pslinks[links_in_pslinks != i]
  connections <-
    rbind(connections,
          data.frame(
            from = rep(i - 1, length(links_in_pslinks)), # -1 for zero-indexing
            to = links_in_pslinks - 1 # here too
            )
          )
}

# results
names(connections) <- c("from", "to")

# make symmetrical
connections <- rbind(connections,
                     data.frame(from = connections$to,
                                to = connections$from))
connections <- connections[!duplicated(connections), ]


## step 4: visualize connections
connections$value <- 1
nodesDF <- data.frame(name = dat$name, group = 1)

network_out <-
  forceNetwork(
    Links = connections,
    Nodes = nodesDF,
    Source = "from",
    Target = "to",
    Value = "value",
    NodeID = "name",
    Group = "group",
    zoom = TRUE,
    opacityNoHover = 3,
    height = 360,
    width = 636
  )

nodesDF$id <- as.numeric(rownames(nodesDF)) - 1
connections_df <-
  merge(connections,
        nodesDF,
        by.x = "to",
        by.y = "id",
        all = TRUE)
to_count_df <- count(connections_df, name)
arrange(to_count_df, desc(n))

## # A tibble: 712 x 2
##    name                     n
##    <fct>                <int>
##  1 Angela Merkel           59
##  2 Andrea Nahles           40
##  3 Heiko Maas              38
##  4 Katarina Barley         38
##  5 Peter Altmaier          38
##  6 Wolfgang Schäuble       38
##  7 Wolfgang Kubicki        37
##  8 Hans-Peter Friedrich    34
##  9 Hermann Gröhe           34
## 10 Ursula von der Leyen    33
## # ... with 702 more rows

# retrieve article titles of interest
enf <- "Europe_of_Nations_and_Freedom"
efdd <- "Europe_of_Freedom_and_Direct_Democracy"

enf_parties <- c(
  "Freedom_Party_of_Austria",
  "Vlaams_Belang",
  "National_Rally_(France)",
  "The_Blue_Party_(Germany)",
  "Lega_Nord",
  "Party_for_Freedom",
  "Congress_of_the_New_Right"
)

efdd_parties <- c(
  "Svobodní",
  "The_Patriots_(France)",
  "Debout_la_France",
  "Alternative_for_Germany",
  "Five_Star_Movement",
  "Order_and_Justice",
  "Liberty_(Poland)",
  "Brexit_Party",
  "Social_Democratic_Party_(UK,_1990–present)",
  "Libertarian_Party_(UK)"
)

articles <- c(enf, efdd, enf_parties, efdd_parties)

# import raw clickstream data
cs <-
  read.table(
    "clickstream-enwiki-2019-05.tsv",
    header = FALSE,
    col.names = c("prev", "curr", "type", "n"),
    fill = TRUE,
    stringsAsFactors = FALSE
  )
cs$n <- as.integer(cs$n)

# subset
cs <- subset(cs, prev %in% articles & curr %in% articles)

# assign categories
cs <- cs %>%
  mutate(
    curr_cat = ifelse(
      curr == enf,
      "ENF Group",
      ifelse(
        curr == efdd,
        "EFDD Group",
        ifelse(curr %in% enf_parties, "ENF Parties",
               "EFDD Parties")
      )
    ),
    prev_cat = ifelse(
      prev == enf,
      "ENF Group",
      ifelse(
        prev == efdd,
        "EFDD Group",
        ifelse(prev %in% enf_parties, "ENF Parties",
               "EFDD Parties")
      )
    )
  )

# summarize data
cs_sum <-  cs %>%
  group_by(curr_cat, prev_cat) %>%
  summarize(n = sum(n)) %>%
  arrange(prev_cat)

# Sankey diagram using plotly
labels <- c(unique(cs_sum$prev_cat), unique(cs_sum$curr_cat))
colors <- ifelse(grepl("EFDD", labels), "#24B9B9", "#2B3856")
sankey_plot <- plot_ly(
  type = "sankey",
  orientation = "h",
  
  node = list(
    label = labels,
    color = colors,
    pad = 15,
    thickness = 15,
    line = list(color = "black",
                width = 0.5)
  ),
  
  link = list(
    source = as.numeric(as.factor(cs_sum$prev_cat)) - 1L,
    target = as.numeric(as.factor(cs_sum$curr_cat)) + 3L,
    value =  cs_sum$n
  ),
  
  height = 340,
  width = 600
) %>%
  layout(font = list(size = 10))

# construct edges
cs_edge <-
  cs %>%
  group_by(prev, curr, prev_cat, curr_cat) %>%
  dplyr::summarise(weight = sum(n)) %>%
  arrange(curr)

# get list of unique articles to construct as nodes
cs_node <- 
  gather(cs_edge,
         `prev`,
         `curr`,
         key = "where",
         value = "article") %>%
  ungroup() %>%
  select(article) %>%
  distinct(article)
names(cs_node) <- c("node")
cs_node$category <-
  ifelse(cs_node$node == enf,
         "ENF Group",
         ifelse(
           cs_node$node == efdd,
           "EFDD Group",
           ifelse(cs_node$node %in% enf_parties, "ENF Parties",
                  "EFDD Parties")
         )
  )

# generate graph
set.seed(3)
cs_net <-
  graph.data.frame(cs_edge,
                   vertices = cs_node,
                   directed = F)
cs_net <-
  igraph::simplify(cs_net, remove.multiple = T, remove.loops = T)

# generate colors based on category
V(cs_net)$color <- 
  ifelse(grepl("EFDD", V(cs_net)$category), "#24B9B9", "#2B3856")

# compute node degrees (#links) and use that to set node size
deg <- igraph::degree(cs_net, mode = "all")
V(cs_net)$size <- deg / 10

# set labels
V(cs_net)$label <- NA
V(cs_net)$label.cex = 0.5
V(cs_net)$label = ifelse(igraph::degree(cs_net) > 5, V(cs_net)$label, NA)
cs_hc_labels <- as.vector(cs_node$node)

# set edge width based on weight
E(cs_net)$width <- log(E(cs_net)$weight) / 5
E(cs_net)$edge.color <- "gray80"

# transform the network to a ggplot friendly format
# (required to generate interactive graph embedded in blog post)
gg_cs_net <-
  ggnetwork(
    cs_net,
    layout = "fruchtermanreingold",
    weights = "weight",
    niter = 50000,
    arrow.gap = 0
  )

cs_plot <- ggplot(gg_cs_net, aes(x = x, y = y, xend = xend, yend = yend)) +
  geom_edges(aes(color = edge.color), size = 0.4, alpha = 0.25) +
  geom_nodes(aes(color = color, size = size)) +
  geom_nodetext(aes(color = color, label = vertex.names, cex = 0.6)) +
  guides(size=FALSE) +
  theme_blank() +
  theme(legend.position = "none")

Collecting Data via Wikidata Queries

Wikidata is a collaboratively edited knowledge base with over 58 million entries as of July 2019. It harbors various types of database items, including text, numerical quantities, coordinates, and images. There are no language editions, but individual entries can have values in different languages.

Wikidata allows users to submit queries using SPARQL, a query language for data stored in RDF (Resource Description Framework) format (see this link). Click here for a brief introduction to SPARQL. While basic queries can be used to answer mundane questions (e.g. “what is the capital city of every member of the European Union, and how many inhabitants live there?”), a targeted combination of related queries can be used for systematic data collection.

Instead of submitting explicit SPARQL queries, the example below uses the WikidataR package to combine various queries in order to collect data on the candidates in the 2019 leadership election of the UK Conservative Party. Suppose we want to retrieve the following information on each candidate:

• name
• sex
• date of birth
• political experience
• education
• official website URL
• Twitter accout
• Facebook account

In Wikidata, entries are stored as items with a unique item ID that starts with “Q”. For instance, the item 2019 Conservative Party (UK) leadership election is stored as “Q30325756”. Items are characterized by a number of statements or claims. Claims start with “P” and detail an item’s properties. For instance, the claim “candidate” is stored as “P726”. Claims have values, which may once again be items. For example, the values of claim “P726” (candidate) of item “Q30325756” (2019 UK Conservative Party leadership election) are 10 items: one entry for each of the 10 candidates running in the leadership election. Take, for example, winning candidate Boris Johnson, who is listed as a candidate under claim “P726”. In turn, the entry on Boris Johnson is stored as item “Q180589”. This item is characterized by numerous claims, including “P1559” (name in native language), “P21” (sex or gender), “P569” (date of birth), “P39” (positions held), “P69” (educated at), “P856” (official website), “P2002” (Twitter username), and “P2013” (Facebook ID).

In order to collect the data for all 10 candidates in the 2019 Conservative Party leadership election, the code chunk below implements the following steps:

1. We retrieve item “Q30325756”, i.e., the entry for 2019 Conservative Party (UK) leadership election
2. We extract claims “P726” of the above item to retrieve the item IDs of all 10 candidates, which we store in the object candidates
3. We save the IDs of the claims of interest, stored in the object claims
4. We then use some nested sapplycommands to do the following:
   • Retrieve the item (entry) for each candidate
   • Extract the eight claims from each candidate item
   • Process the informational value of each extracted claim, depending on whether the claim value is
     • an atomic object (such as web site URLs)
     • a textual object with auxiliary information (such as names, which come with language information)
     • a time/date (such as date of birth)
     • yet another item (such as previous positions, where each position has an own data base entry)

# get item based on item id
uk_item <- get_item("Q30325756", language = "en")

# extract candidates
candidates <- extract_claims(uk_item, claims = "P726")
candidates <- candidates[[1]][[1]]$mainsnak$datavalue$value$id

# collect the following attributes ("claims") for each candidate
claims <- c("P1559", "P21", "P569", "P39", "P69", "P856", "P2002", "P2013")
names(claims) <- c("nam", "sex", "dob", "exp", "edu", "web", "twi", "fbk")
claims

##     nam     sex     dob     exp     edu     web     twi     fbk 
## "P1559"   "P21"  "P569"   "P39"   "P69"  "P856" "P2002" "P2013"

# retrieve data
uk_data <-
  sapply(candidates,
         function (item) {
           tmp_item <- get_item(item, language = "en")
           sapply(claims,
                  function(claim) {
                    tmp_claim <- extract_claims(tmp_item, claim)[[1]][[1]]
                    if (any(is.na(tmp_claim))) {
                      return(NA)
                    } else {
                      tmp_claim <- tmp_claim$mainsnak$datavalue$value
                      if (is.atomic(tmp_claim)) {
                        return(tmp_claim)
                      } else if ("text" %in% names(tmp_claim)) {
                        return(tmp_claim$text)
                      } else if ("time" %in% names(tmp_claim)) {
                        tmp_claim <- as.Date(substr(tmp_claim$time, 2, 11))
                        return(tmp_claim)
                      } else if ("id" %in% names(tmp_claim)) {
                        tmp_claim <- tmp_claim$id
                        tmp_claim <- 
                          sapply(tmp_claim, 
                                 get_item, 
                                 language = "en",
                                 simplify = FALSE,
                                 USE.NAMES = TRUE)
                        tmp_claim <-
                          sapply(tmp_claim,
                                 function (x) {
                                   x[[1]]$labels$en$value
                                 })
                        return(tmp_claim)
                      }
                    }
                  },
                  simplify = FALSE,
                  USE.NAMES = TRUE)
         },
         simplify = FALSE,
         USE.NAMES = TRUE
  )

The retrieved data are stored in a nested list. At the upper level of the list, we have the ten candidates, named with their respective item IDs. Nested within each of the ten upper-level elements, we have the values of the eight claims, named with the labels we specified above. Claim values may either be atomic (such as date of birth) or vectors (such as “positions held”, which may have multiple entries). Below, we can see the retrieved data for the first candidate on the list, winning candidate Boris Johnson.

## $nam
## [1] "Boris Johnson"
## 
## $sex
## Q6581097 
##   "male" 
## 
## $dob
## [1] "1964-06-19"
## 
## $exp
##                                                    Q38931 
##                                         "Mayor of London" 
##                                                  Q1371091 
## "Secretary of State for Foreign and Commonwealth Affairs" 
##                                                 Q28841847 
##       "Member of the Privy Council of the United Kingdom" 
##                                                 Q30524710 
##     "Member of the 57th Parliament of the United Kingdom" 
##                                                 Q30524718 
##     "Member of the 56th Parliament of the United Kingdom" 
##                                                 Q35647955 
##     "Member of the 54th Parliament of the United Kingdom" 
##                                                 Q35921591 
##     "Member of the 53rd Parliament of the United Kingdom" 
##                                                    Q14211 
##                    "Prime Minister of the United Kingdom" 
##                                                  Q3303456 
##                        "Leader of the Conservative Party" 
##                                                 Q77685926 
##     "Member of the 58th Parliament of the United Kingdom" 
##                                                   Q609884 
##                              "First Lord of the Treasury" 
##                                                  Q3315116 
##                          "Minister for the Civil Service" 
##                                                 Q65988624 
##                                  "Minister for the Union" 
## 
## $edu
##                       Q192088                       Q805285 
##                "Eton College"             "Balliol College" 
##                      Q4804780                      Q5413121 
##        "Ashdown House School" "European School, Brussels I" 
## 
## $web
## [1] "http://www.boris-johnson.com"
## 
## $twi
## [1] "BorisJohnson"
## 
## $fbk
## [1] "borisjohnson"

legislatoR

legislatoR is a joint project of Sascha Göbel and Simon Munzert. It offers a comprehensive relational individual-level database that provides political, sociodemographic, and other Wikipedia-related data on members of various national parliaments, including the all sessions of the Austrian Nationalrat, the German Bundestag, the Irish Dáil, the French Assemblée, and the United States Congress (House and Senate). It currently comprises data of 42,534 elected representatives and holds information for a wide variety of variables, including:

• sociodemographics (Core)
• basic political variables (Political)
• records of individual Wikipedia data, including full revision histories (History)
• daily user traffic on individual Wikipedia biographies (Traffic)
• social media handles and website URLs (Social)
• URLs to individual Wikipedia portraits (Portraits)
• information on public offices held by MPs (Offices)
• MPs’ occupations (Professions)
• IDs that link politicians to other files, databases and websites (IDs)

The figure below, taken from Göbel and Munzert (2019), illustrates the data structure:

The package provides a relational database. This means that all data sets can be joined with the core data set via one of two keys: the Wikipedia page ID or the Wikidata ID, which uniquely identify individual politicians.

legislatoR services the increasing demand for micro-level data on political elites among political scientists, political analysts, and journalists and offers an accessible and rich collection of data on past and present politicians. The inclusion of Wikipedia and other web data allows for the inclusion of detailed information on politicians’ biographies.

To install the current developmental version from GitHub, we use the devtools package. After installing and loading legislatoR, we can use the ls() command to explore the full functionality of the package.

## Install from GitHub
devtools::install_github("saschagobel/legislatoR")
library(legislatoR)

## View functionality
ls("package:legislatoR")

## Get social media adoption rates
# get data: Germany
dat_ger <- right_join(
  x = get_core(legislature = "deu"),
  y = filter(
    get_political(legislature = "deu"),
    as.numeric(session) == max(as.numeric(session))
  ),
  by = "pageid"
)
dat_ger <- left_join(x = dat_ger,
                     y = get_social(legislature = "deu"),
                     by = "wikidataid")
dat_ger$legislature <- "Germany"
dat_ger_sum <- dat_ger %>%
  dplyr::summarize(
    twitter = mean(not(is.na(twitter)), na.rm = TRUE),
    facebook = mean(not(is.na(facebook)), na.rm = TRUE),
    website = mean(not(is.na(website)), na.rm = TRUE),
    session_start = ymd(first(session_start)),
    session_end = ymd(first(session_end)),
    legislature = first(legislature)
  )

##     twitter  facebook  website session_start session_end legislature
## 1 0.7591036 0.6918768 0.640056    2017-10-24  2021-10-24     Germany

Application 2: Public Attention to Members of the German Bundestag

In the second application, we use pageviews to identify peaks in public attention for MPs over time. This is particularly interesting in the context of politically significant events. For instance, we may want to know about public attention to parliamentarians following scandals, around elections, or during election campaigns. The code below illustrates this logic by averaging daily pageviews across all Wikipedia articles on members of the German Bundestag between July 2015 and December 2017.

Code: Plotting Average Daily Pageviews for German MPs

## Visualize average pageviews data of German MPs
# get data
ger_traffic <- right_join(
  x = get_traffic(legislature = "deu"),
  y = filter(
    get_political(legislature = "deu"),
    session_end >= as.Date("2015-07-01")
  ),
  by = "pageid"
)
ger_traffic <- left_join(x = ger_traffic,
                         y = get_core(legislature = "deu"),
                         by = "pageid")
ger_traffic <-
  dplyr::select(ger_traffic, pageid, date, traffic, session, party, name)

# aggregate data
ger_traffic$date <- ymd(ger_traffic$date)
ger_traffic_date <- group_by(ger_traffic, date)
ger_traffic_legislators <- group_by(ger_traffic, pageid)
ger_traffic_sum <-
  summarize(ger_traffic_date, mean = mean(traffic, na.rm = TRUE))
ger_traffic_sum <- mutate(
  ger_traffic_sum,
  mean_l1 = lag(mean, 1),
  mean_f1 = lead(mean, 1),
  peak = (mean >= 1.8 * mean_l1 &
            mean > 180)
)

# identify peaks
ger_traffic_peaks <- filter(ger_traffic_sum, peak == TRUE)
ger_traffic_peaks_df <-
  filter(ger_traffic, date %in% ger_traffic_peaks$date)
ger_traffic_peaks_group <-
  group_by(ger_traffic_peaks_df, date) %>% 
  dplyr::arrange(desc(traffic)) %>% 
  filter(row_number() == 1)
ger_traffic_peaks_group <- arrange(ger_traffic_peaks_group, date)
events_vec <-
  c(
    "deceased",
    "drug affair",
    "bullying affair",
    "candidacy for presidency",
    "deceased",
    "???",
    "chancellorchip announcement",
    "elected president",
    "???",
    "policy success",
    "TV debate",
    "general election",
    "threat to resign",
    "elected speaker of parliament"
  )

# plot
par(oma = c(0, 0, 0, 0))
par(mar = c(0, 4, 0, .5))
par(yaxs = "i", xaxs = "i", bty = "n")
layout(matrix(c(1, 1, 3, 2, 2, 3), 2, 3, byrow = TRUE),
       heights = c(1, 2, 3),
       widths = c(5, 5, 1.8))
# names labels
plot(
  ymd(ger_traffic_sum$date),
  rep(0, length(ger_traffic_sum$date)),
  xlim = c(ymd("2015-07-01"), ymd("2018-01-01")),
  xaxt = "n",
  ylim = c(0, 8),
  yaxt = "n",
  xlab = "",
  ylab = "",
  cex = 0
)
text(
  ger_traffic_peaks_group$date,
  0,
  ger_traffic_peaks_group$name,
  cex = .75,
  srt = 90,
  adj = c(0, 0)
)
# pageviews time series
par(mar = c(2, 4, 0, .5))
plot(
  ymd(ger_traffic_sum$date),
  ger_traffic_sum$mean,
  type = "l",
  ylim = c(0, 1.25 * max(ger_traffic_sum$mean)),
  xlim = c(ymd("2015-07-01"), ymd("2018-01-01")),
  xaxt = "n",
  yaxt = "n",
  xlab = "",
  ylab = "mean(pageviews)",
  col = "white"
)
abline(h = seq(0, 1.5 * max(ger_traffic_sum$mean), 250), col = "lightgrey")
lines(ymd(ger_traffic_sum$date), ger_traffic_sum$mean, lwd = .5)
dates <- seq(ymd("2015-07-01"), ymd("2018-01-01"), by = 1)
axis(1, dates[day(dates) == 1 &
                month(dates) %in% c(1, 4, 7, 10)], labels = FALSE)
axis(1,
     dates[day(dates) == 1 &
             month(dates) %in% c(1)],
     lwd = 0,
     lwd.ticks = 3,
     labels = FALSE)
axis(1,
     dates[day(dates) == 1 &
             month(dates) %in% c(7)],
     labels = as.character(year(dates[day(dates) == 15 &
                                        month(dates) %in% c(7)])),
     tick = F,
     lwd = 0)
axis(2, seq(0, 1.5 * max(ger_traffic_sum$mean), 250), las = 2)
# events labels in time series
for (i in seq_along(events_vec)) {
  text(ger_traffic_peaks_group$date[i],
       ger_traffic_sum$mean[ger_traffic_sum$peak == TRUE][i] + 80,
       i,
       cex = .8)
  points(
    ger_traffic_peaks_group$date[i],
    ger_traffic_sum$mean[ger_traffic_sum$peak == TRUE][i] + 80,
    pch = 1,
    cex = 2.2
  )
}
# election date
# events labels explained
par(mar = c(0, 0, 0, 0))
plot(
  0,
  0,
  xlim = c(0, 5),
  ylim = c(0, 10),
  xaxt = "n",
  yaxt = "n",
  xlab = "",
  ylab = "",
  cex = 0
)
positions <-
  data.frame(
    events_xpos = 0.45,
    events_ypos = seq(6.5, (6.5 - .5 * length(events_vec)),-.5),
    text_xpos = .5
  )
text(0, 7, "Events", pos = 4, cex = .75)
for (i in seq_along(events_vec)) {
  text(positions$events_xpos[i], positions$events_ypos[i], i, cex = .8)
  points(positions$events_xpos[i],
         positions$events_ypos[i],
         pch = 1,
         cex = 2.2)
  text(
    positions$text_xpos[i],
    positions$events_ypos[i],
    events_vec[i],
    pos = 4,
    cex = .75
  )
}

The plot shows 14 notable spikes in daily pageviews. We identify which politician’s article generated the most traffic during each of these events. Furthermore, we add a legend that lists salient political events that likely caused these spikes in attention to MPs Wikipedia entries. For example, spike 14 marks the day on which Wolfgang Schäuble (CDU) was elected speaker of the parliament.

Conclusion

Collecting and analyzing Wikipedia data is relatively easy and entirely free. It enables researchers to use and analyze an enormous body of data that offers valuable information for research in political science and beyond. Tools that facilitate the collection, processing, and analysis of Wikipedia data advance rapidly, broadening the realm of possibilities for scientific research. Political science research is increasingly picking up on these developments, as is evident in recent contributions (Munzert 2015; Göbel and Munzert 2018; Shi et al. 2019) and softwares (such as legislatoR).

However, using Wikipedia data may also come with limitations and pitfalls. As entries can be read and edited by both humans and machines, the accuracy of contents and the validity of metadata are not guaranteed. With respect to the latter, Wikidata adds provenance information to all the data. These can be used to evaluate the validity of the data in question for applied research. Researchers should also keep in mind that Wikipedia data highly depends on user-driven creation, editing, and use of contents. This may not only lead to systematic selection bias due to data availability but also induce problems of equivalence of data points (e.g., articles on historical political figures likely receive fewer views and edits than articles on active politicians for reasons unrelated to their legislative activity or real-world importance).

These caveats are however all but exclusive to Wikipedia data. They merely underline that Wikipedia data is no exception when it comes to the general necessity of thoroughly scrutinizing and critically assessing the suitability of any given data for addressing substantive research questions.

About the Presenter

Simon Munzert is a lecturer in Political Data Science at the Hertie School of Governance in Berlin, Germany. A former member of the MZES Data and Methods Unit, Simon founded the Social Science Data Lab in 2016. His research focuses on public opinion, political representation, and the role of new media for political processes.

A tour of Quantitative Text Analysis

The workshop started with a few basics: While QTA can be efficient and cheap, it always fails to rely on a correct model of language. It does not free us from reading texts, and validation is key. We learned about the basic distinction between classification (organizing text into categories) and scaling (estimation positions of actors), and its supervised (where hand-coded or other external data is available) and unsupervised (without such data) variants.

(Pre-)processing text

We relied on the R package quanteda developed by Ken Benoit and collaborators, which takes QTA by storm, at least for those working in R. Together with the readtext package, it easily allows to get your text data into R, create a so-called “corpus” of texts with the actual content as well as meta-information, and perform various tasks of corpus and text processing (subsetting a corpus, creating a document-feature matrix (dfm), stopword removal) as well as analysis (scaling, classification). A large and increasing number of extras is also available, such as ways to assess text similarity (function textsta_simil()) and lexical diversity (textstat_lexdiv()). Some of these features are demonstrated in an example below. Also see this overview by quanteda for a full list of functions and the preText package for advise on evaluating pre-processing specifications.

A small coalition corpus

For a few examples from the workshop, consider a small set of three documents: The coalition agreement between the CDU/CSU and the SPD as well as the respective election manifestos from the 2017 Bundestag election. The corpus is created by:

library(readtext)
library(quanteda)
text <- readtext(paste0(wd, "coalition/*.txt"),
                 docvarsfrom = "filenames",
                 docvarnames = "Party")
text$text <- gsub("\n", " ", text$text)
coalitioncorpus <- corpus(text, docid_field = "doc_id")
coalitioncorpus$metadata$source <- "[directory] on [system] by [user]"
summary(coalitioncorpus)

## Corpus consisting of 3 documents:
## 
##           Text Types Tokens Sentences     Party
##     cducsu.txt  4738  26004      1288    cducsu
##  coalition.txt 11660  93214      3763 coalition
##        spd.txt  7650  50298      2402       spd
## 
## Source: [directory] on [system] by [user]
## Created: Wed Nov 11 15:06:41 2020
## Notes:

The code relies on the two packages, readtext and quanteda, to create a data frame with the text files, using their names for a document-level variable called “Party”. The gsub() command removes whitespace, and corpus() turns the data frame into a corpus, which is a special case of a data frame containing texts, some meta-information and document-level variables, all optimized to perform a variety of quantitative text analysis operations using quanteda.

Further document-level variables are added via:

docvars(coalitioncorpus, "Year") <- 2017
docvars(coalitioncorpus, "Party_regex") <- 
  sub("[\\.].*", "", names(texts(coalitioncorpus)))
docvars(coalitioncorpus)

##                   Party Year Party_regex
## cducsu.txt       cducsu 2017      cducsu
## coalition.txt coalition 2017   coalition
## spd.txt             spd 2017         spd

Note that this uses a regular expression (regex) to alternatively retrieve the party names from the filenames after creating the corpus. For specific analyses, we want to know the distribution of words across documents and create a document-feature matrix (dfm):

dfm_coal <- dfm(
  coalitioncorpus,
  remove = c(stopwords("german"),
             "dass",
             "sowie",
             "insbesondere"),
  remove_punct = TRUE,
  stem = FALSE
)
dfm_coal[, 1:8]

## Document-feature matrix of: 3 documents, 8 features (16.7% sparse).
## 3 x 8 sparse Matrix of class "dfm"
##                features
## docs            gutes land zeit deutschland liebens lebenswertes gut
##   cducsu.txt        6   48   11         147       1            1  16
##   coalition.txt     1   39   14         195       0            0  13
##   spd.txt           6   44   33          97       0            0  17
##                features
## docs            wohnen
##   cducsu.txt         1
##   coalition.txt     10
##   spd.txt            6

Creating a dfm induces a bag-of-words assumption. This means that the order in which words appear is ignored. A dfm is a means of information reduction and the most efficient way of storing text as data, but allows only analyses under this assumption. We quickly glance at the similarity (function textstat_simil()) and lexical diversity (function textstat_lexdiv()) of texts:

simil <- textstat_simil(dfm_coal,
                        margin = "documents",
                        method = "correlation")
simil 

## textstat_simil object; method = "correlation"
##               cducsu.txt coalition.txt spd.txt
## cducsu.txt         1.000         0.968   0.975
## coalition.txt      0.968         1.000   0.986
## spd.txt            0.975         0.986   1.000

textstat_lexdiv(dfm_coal)[, 1:2]

##        document       TTR
## 1    cducsu.txt 0.3302084
## 2 coalition.txt 0.2487993
## 3       spd.txt 0.2802713

From this, we learn that the SPD manifesto has more similarity to the coalition agreement than that of the CDU/CSU, a notion which somewhat resembles the assessment of the 2017 German coalition. Also, the lexical diversity of the manifestos, measured in terms of types (different words) per token (total words), appears to be higher than the coalition agreement, especially for the CDU/CSU.

Sentiment analysis using a dictionary

For sentiment analyis, existing dictionaries are available. It is important to note that these do not necessarily fit the research question at hand. In this example, the German “LIWC” (linguistic inquiry and word count) dictionary is used, but alternatives exist, such as “Lexicoder” for political text. LIWC features the categories “anger”, “posemo” (positive emotion) and “religion”. After obtaining the dictionary and applying it while creating a dfm from the corpus, the share of the texts in the respective categories is displayed. The results indicate that the coalition agreement features less positive emotions as compared to the manifestos and that the SPD manifesto is the most “angry” text, while the CDU/CSU speaks most about religion.

# Create dictionary
liwcdict <- dictionary(file = paste0(wd, "German_LIWC2001_Dictionary.dic"),
                       format = "LIWC")

# Create dfm
liwcdfm <- dfm(
  coalitioncorpus,
  remove = c(stopwords("german")),
  remove_punct = TRUE,
  stem = FALSE,
  dictionary = liwcdict
)

# Subset and calculate percentage
liwcsub <-
  dfm_select(liwcdfm,
             pattern = c("Anger", "Posemo", "Relig"),
             selection = "keep")

liwcsub <- convert(liwcsub, to = "data.frame")
liwcsub$sum <- apply(dfm_coal, FUN = sum, 1)
liwcparties <- data.frame(
  docs = liwcsub$document,
  ShareAnger = liwcsub$Anger / liwcsub$sum,
  SharePosemo = liwcsub$Posemo / liwcsub$sum,
  ShareRelig = liwcsub$Relig / liwcsub$sum
)

liwcparties 

##            docs  ShareAnger SharePosemo  ShareRelig
## 1    cducsu.txt 0.004314995  0.06414959 0.005609493
## 2 coalition.txt 0.004666188  0.05050462 0.004074997
## 3       spd.txt 0.005550042  0.05697063 0.003454993

LDA topic modelling

LDA stands for Latent Dirichlet allocation. In a nutshell, the method represents texts as a mixture of topics, and simultaneously topics as mixtures of words. Fixing the number of topics to \(k = 5\), and using the topicmodels package, the command lda() delivers posterior probabilities of the topics for each document.

library(topicmodels)

# Preparation
dfm_coal <-
  dfm(
    coalitioncorpus,
    remove = c(
      stopwords("german"),
      "dass",
      "sowie",
      "insbesondere",
      "b",
      "z",
      "a",
      "u"
    ),
    remove_punct = TRUE,
    remove_numbers = TRUE,
    stem = FALSE
  )

dfm_coal <- dfm_wordstem(dfm_coal, language = "german")

# Define parameters
burnin <- 1000
iter <- 500
keep <- 50
seed <- 2010
ntopics <- 5

# Run LDA with 5 topics
ldaOut <- LDA(
  dfm_coal,
  k = ntopics,
  method = "Gibbs",
  control = list(
    burnin = burnin,
    iter = iter,
    keep = keep,
    seed = seed,
    verbose = FALSE
  )
)
# Posterior probabilities of the topics for each document
k <- posterior(ldaOut)

Interpretation is the difficult part. Each document can be expressed as a mixture of topics, and notwithstanding the precise meaning of the topics, we learn that all texts share content referring to topic 1, while the CDU/CSU manifesto also features topic 2 and the SPD manifesto topic 3 to some extent.

k$topics

##                       1          2          3          4          5
## cducsu.txt    0.5815482 0.02729874 0.36122496 0.01144536 0.01848272
## coalition.txt 0.6950774 0.03700023 0.06163624 0.10570834 0.10057777
## spd.txt       0.6755292 0.17646590 0.11404313 0.01837605 0.01558576

Wordfish Scaling

Wordfish scaling derives latent positions from texts based on a bag-of-words assumption. Here is an example relying on a set of Swiss manifestos, using only the sections on immigration. In preparation, a dfm is created while removing stopwords, stemming the remaining words and removing punctuation.

manifestos <- readtext(paste0(wd, "manifestos/*.txt"))
manifestocorpus <- corpus(manifestos)
dfm_manifesto <-
  dfm(
    manifestocorpus,
    remove = c(
      "gruen*",
      "sp",
      "sozialdemokrat*",
      "cvp",
      "fdp",
      "svp",
      "fuer",
      "dass",
      "koennen",
      "koennte",
      "ueber",
      "waehrend",
      "wuerde",
      "wuerden",
      "schweiz*",
      "partei*",
      stopwords("german")
    ),
    valuetype = "glob",
    stem = FALSE,
    remove_punct = TRUE
  )
dfm_manifesto <- dfm_wordstem(dfm_manifesto, language = "german")

The function textmodel_wordfish() computes the Wordfish model, originally decribed in an AJPS article by Slapin and Proksch in 2009. It assumes that the distribution of words across texts follows a Poisson distribution, and can be modeled by document and word fixed effects as well as word-specific weights and document positions. The model is a variant of unsupervised scaling, only requiring the relative location of two texts on the latent dimension. Here, a text of the Swiss People’s Party (SVP) is assumed to be right to that of the Social Democratic Party of Switzerland (SPS). The results make some sense, with the other manifestos aligning as expected on what could be interpreted as a anti-immigration dimension.

wf <- textmodel_wordfish(dfm_manifesto,
                         dir = c(13, 19),
                         dispersion = "poisson")
textplot_scale1d(wf)

Or, with some improvements to the plot:

library(ggplot2)

# Save document scores and confidence intervals in data frame
wfdata <- as.data.frame(predict(wf, interval = "confidence"))

# Add document variables
wfdata$docs <- rownames(wfdata)
wfdata$electionyear <- substr(wfdata$docs, 5, 8)
wfdata$party <- as.factor(substr(wfdata$docs, 1, 3))
wfdata$party <-
  factor(wfdata$party, levels = c("gps", "sps", "cvp", "fdp", "svp"))

ggplot(wfdata) +
  geom_pointrange(
    aes(
      x = electionyear,
      y = fit.fit,
      ymin = fit.lwr,
      ymax = fit.upr,
      group = party,
      color = party
    ),
    size = 0.5
  ) +
  geom_line(aes(
    x = electionyear,
    y = fit.fit,
    group = party,
    color = party
  )) +
  theme_bw() +
  labs(title = "Wordfish analysis",
       y = "Document position",
       x = "Electionyear") +
  scale_color_manual(values = c("green3",
                                "red1",
                                "darkorange1",
                                "dodgerblue4",
                                "springgreen4"))

Further readings

• An introductory article to QTA in R, especially relying on quanteda: Welbers, Kasper, Wouter Van Atteveldt and Kenneth Benoit (2017): Text Analysis in R, Communication Methods and Measures 11(4): 245-65.
• An application of the methods by the workshop host and co-authors: Schwarz, Daniel, Denise Traber and Kenneth Benoit (2017): Estimating Intra-Party Preferences: Comparing Speeches to Votes. Political Science Research and Methods 5(2): 379-396.

About the presenter

Denise Traber is a Senior Research Fellow at the University of Lucerne, Switzerland, where she heads an Ambizione research grant project on “The divided people: polarization of political attitudes in Europe” funded by the Swiss National Science Foundation. She has a strong interest in quantitative text analysis, co-organizes the “Zurich Summer School for Women in Political Methodology” and has published the article “Estimating Intra-Party Preferences: Comparing Speeches to Votes” in Political Science Research and Methods in 2017, jointly with Daniel Schwarz and Ken Benoit.

About Twitter

Twitter is an online news and social networking service, also used for micro-blogging. In everyday use, it mostly serves as a platform for publicly sharing short texts – often along with media content and/or links – in the form of so-called “tweets”. Twitter has approx. 326 Million monthly active users who send about 500 Million tweets each day (see this fact sheet).

Twitter in Social Science Research

Analyzing tweets and social interaction on Twitter can help to answer social science research questions, especially in communication research and political science. Contrary to Facebook (API depreciation/shut-down in April 2018) and Instagram (API depreciation/shut-down in December 2018), Twitter data is (easily) accessible for researchers.

According to the Web of Science database, there are 2,598 journal articles published since 2009 with the word “Twitter” in their titles.

Figure 1: Number of articles by year

Figure 2: Number of articles by discipline

Collecting Twitter Data

# Install Packages
install.packages("rtweet")
install.packages("ggmap")
install.packages("igraph")
install.packages("ggraph")
install.packages("tidytext")
install.packages("ggplot2")
install.packages("dplyr")
install.packages("readr")

# Set Working Directory
setwd("/PATH")

# Open Libraries
library(rtweet)

# Speficy Authentification Token's provided in your Twitter App
create_token(
  app = "APPNAME",
  consumer_key = "consumer_key",
  consumer_secret = "consumer_secret",
  access_token = "access_token",
  access_secret = "access_secret"
)

# Collect a 'random' sample of Tweets for 10 seconds
stream_tweets(
  q = "",
  timeout = 10,
  file_name = "sample.json",
  parse = FALSE
)
sample <- parse_stream("sample.json")
save(sample, file = "sample_live.Rda")

# Collect Tweets that contain specific keywords for 30 seconds
stream_tweets(
  q = "trump, donald trump",
  timeout = 30,
  file_name = "trump.json",
  parse = FALSE
)
trump <- parse_stream("trump.json")
save(trump, file = "trump_live.Rda")

# Collect Tweets from a specific location
stream_tweets(
  c(13.0883,52.3383,13.7612,52.6755), 
  timeout = 180,
  file_name = "berlin.json",
  parse = FALSE
)
berlin <- parse_stream("berlin.json")
save(berlin, file = "berlin_live.Rda")

# Dimensions of the data frame
dim(sample)

## [1] 359  88

# Variables in the data frame
names(sample)

##  [1] "user_id"                 "status_id"              
##  [3] "created_at"              "screen_name"            
##  [5] "text"                    "source"                 
##  [7] "display_text_width"      "reply_to_status_id"     
##  [9] "reply_to_user_id"        "reply_to_screen_name"   
## [11] "is_quote"                "is_retweet"             
## [13] "favorite_count"          "retweet_count"          
## [15] "hashtags"                "symbols"                
## [17] "urls_url"                "urls_t.co"              
## [19] "urls_expanded_url"       "media_url"              
## [21] "media_t.co"              "media_expanded_url"     
## [23] "media_type"              "ext_media_url"          
## [25] "ext_media_t.co"          "ext_media_expanded_url" 
## [27] "ext_media_type"          "mentions_user_id"       
## [29] "mentions_screen_name"    "lang"                   
## [31] "quoted_status_id"        "quoted_text"            
## [33] "quoted_created_at"       "quoted_source"          
## [35] "quoted_favorite_count"   "quoted_retweet_count"   
## [37] "quoted_user_id"          "quoted_screen_name"     
## [39] "quoted_name"             "quoted_followers_count" 
## [41] "quoted_friends_count"    "quoted_statuses_count"  
## [43] "quoted_location"         "quoted_description"     
## [45] "quoted_verified"         "retweet_status_id"      
## [47] "retweet_text"            "retweet_created_at"     
## [49] "retweet_source"          "retweet_favorite_count" 
## [51] "retweet_retweet_count"   "retweet_user_id"        
## [53] "retweet_screen_name"     "retweet_name"           
## [55] "retweet_followers_count" "retweet_friends_count"  
## [57] "retweet_statuses_count"  "retweet_location"       
## [59] "retweet_description"     "retweet_verified"       
## [61] "place_url"               "place_name"             
## [63] "place_full_name"         "place_type"             
## [65] "country"                 "country_code"           
## [67] "geo_coords"              "coords_coords"          
## [69] "bbox_coords"             "status_url"             
## [71] "name"                    "location"               
## [73] "description"             "url"                    
## [75] "protected"               "followers_count"        
## [77] "friends_count"           "listed_count"           
## [79] "statuses_count"          "favourites_count"       
## [81] "account_created_at"      "verified"               
## [83] "profile_url"             "profile_expanded_url"   
## [85] "account_lang"            "profile_banner_url"     
## [87] "profile_background_url"  "profile_image_url"

# An anonymized portion of the data frame, only tweets in English
sample$user_id <- seq(1, nrow(sample), 1)
sample$screen_name <- paste("name", seq(1, nrow(sample), 1), sep = "_")
sample <- subset(sample, lang == "en")
sample[1:5, c("user_id", "created_at", "screen_name", "text", "is_quote", 
              "is_retweet")]

## # A tibble: 5 x 6
##   user_id created_at          screen_name text          is_quote is_retweet
##     <dbl> <dttm>              <chr>       <chr>         <lgl>    <lgl>     
## 1       1 2019-04-03 10:22:32 name_1      just see..<U+2764><U+FE0F>~ FALSE    FALSE     
## 2       4 2019-04-03 10:22:33 name_4      I’m slowy gi~ FALSE    TRUE      
## 3      15 2019-04-03 10:22:33 name_15     "\"I didn't ~ TRUE     FALSE     
## 4      20 2019-04-03 10:22:33 name_20     Hmm. I’ma de~ FALSE    TRUE      
## 5      28 2019-04-03 10:22:33 name_28     I still hear~ TRUE     TRUE

Analyzing Twitter Data

We have now collected Tweets and meta-information based on selected keywords and/or regional parameters. This begs the question what to do with the raw data. Some common research interests include:

• Content Analysis: What kind of topics are users talking about?
• Sentiment Analysis: What kind of opinions, attitudes, and emotions towards objects are users communicating?
• Network Analysis: Who is related to whom? Who are important users?
• Geospatial Analysis: Where are users/Tweets coming from?

In the following, we present two quick examples that showcase possible avenues for the analysis of Twitter data.

Example 1: Prepping Tweets for Text Analysis

Quantitative Text Analysis (QTA) typically relies on pre-processed textual data (see this blog post for our SSDL workshop on QTA by Denise Traber). The code chunk below illustrates how a collection of Tweets can easily be prepared for QTA techniques such as content or sentiment analysis.

In this example, we use our 30 second sample of Tweets containing the key words “donald trump” and/or “trump”, which we stored as trump_live.RDa (see Collecting Tweets Using the rtweet Package above). Tweet contents are stored in the variable trump$text. Using the tidytextand dplyr packages, we then process the Tweets into a text corpus as follows:

1. Remove URLs from all tweets using gsub()
2. Remove punctuation, convert to lowercase, and seperate all words using unnest_tokens()
3. Remove stop words by first loading a list of stop words from the tidytext package via data("stop_words") and then removing these words from our tweets via anti_join(stop_words)

Code: Preparing Data for Content Analysis

# Open Libraries
library(tidytext)
library(dplyr)

# Data Cleaning
# Delete Links in the Tweets
trump$text <- gsub("http.*", "", trump$text)
trump$text <- gsub("https.*", "", trump$text)
trump$text <- gsub("&amp;", "&", trump$text)

# Remove punctuation, convert to lowercase, seperate all words
trump_clean <- trump %>%
  dplyr::select(text) %>%
  unnest_tokens(word, text)

# Load list of stop words - from the tidytext package
data("stop_words")

# Remove stop words from your list of words
cleaned_tweet_words <- trump_clean %>%
  anti_join(stop_words)

Following this, we can calculate the word counts in our Tweet collection and visualize the 15 most frequent words:

Code: Plotting the 15 Most Frequent Words

# Plot the top 15 words
cleaned_tweet_words %>%
  count(word, sort = TRUE) %>%
  top_n(15) %>%
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(x = word, y = n)) +
  geom_col() +
  xlab(NULL) +
  coord_flip() +
  labs(y = "Count",
       x = "Unique words",
       title = "Count of unique words found in tweets",
       subtitle = "Stop words removed from the list")

## Selecting by n

Figure 3: Top 15 Most Frequent Words

Example 2: Analyzing Locations

Provided that users share their exact geo-information, we can locate their tweets on geographical maps. For example, in our 3 minute sample of Tweets from berlin (see Collecting Tweets Using the rtweet Package above), this information was available for only two Tweets.

The code below demonstrates how one can view these Tweets mapped onto their exact geographical locations. We first pre-process the data, extracting numerical geo-coordinates for longitude and latitude where this information is available. We then subset the data frame to observations where these information exist. Lastly, using the same geo-coordinates as in the data collection, we set up an empty rectangular map of Berlin. Note that the last step involves accessing the Google Maps API, for which you will have to register separately (see, e.g., this post at R-bloggers).

Code: Analyzing Locations - Setup

library(ggmap)
library(dplyr)

# Seperate Geo-Information (Lat/Long) Into Two Variables
berlin <- tidyr::separate(data = berlin,
                          col = geo_coords,
                          into = c("Latitude", "Longitude"),
                          sep = ",",
                          remove = FALSE)

# Remove Parentheses
berlin$Latitude <- stringr::str_replace_all(berlin$Latitude, "[c(]", "")
berlin$Longitude <- stringr::str_replace_all(berlin$Longitude, "[)]", "")

# Store as numeric
berlin$Latitude <- as.numeric(berlin$Latitude)
berlin$Longitude <- as.numeric(berlin$Longitude)

# Keep only those tweets where geo information is available
berlin <- subset(berlin, !is.na(Latitude) & !is.na(Longitude))

# Set up empty map
berlin_map <- get_map(location = c(lon = mean(c(13.0883, 13.7612)),
                                   lat = mean(c(52.3383, 52.6755))),
                      zoom = 10,
                      maptype = "terrain",
                      source = "google")

Building up on this, we can then map our collected tweets to see from where they were sent: Berlin-Mitte and Potsdam.

Code: Analyzing Locations - Map

tweet_map <- ggmap(berlin_map)
tweet_map + geom_point(data = berlin,
                       aes(x = Longitude,
                           y = Latitude),
                           color = "red",
                           size = 4,
                           alpha = .5)

Figure 4: GPS Coordinates of 2 Tweets

Potential Issues and Challenges

Conclusion: Twitter in the Social Sciences

Collecting Twitter data is comparatively easy and cheap. However, we usually know close to nothing about the users whom we collect tweets from. Thus, the research potential for social science projects is very limited when we are interested in questions that address ‘outside-social-media’ phenomena.

Arguably, we benefit most from Twitter data when we treat it as auxiliary or proxy information, or when we use it in combination with other data sources. Potential applications along those lines include:

• Variation in fear of crime across different regions
• Health monitoring over time
• Monitoring highly discussed topics in realtime
• Measuring existing stereotypes towards minorities

Further Reading

• Workshop on using rtweet by its developer Michael W. Kearney
• rtweet documentation
• Test mining of tweets
• Using Shiny and Leaflet
• Rieder, Y. & S. Kühne, 2018: Geospatial Analysis of Social Media Data - A Practical Framework and Applications. In: Stuetzer, C.M., Welker, M. & M. Egger (Eds.), Computational Social Science in the Age of Big Data. Concepts, Methodologies, Tools, and Applications. DGOF Schriftenreihe, Köln: Herbert van Halem Verlag. URL: http://www.halem-verlag.de/computational-social-science-in-the-age-of-big-data/.

About the Presenter

Simon Kühne is a post-doc at Bielefeld University. He holds a BA in Sociology and an MA in Survey Methodology from the University of Duisburg-Essen and a PhD in Sociology from Humboldt University of Berlin. His research focuses on survey methodology, social media and online data, and social inequality.