<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Survey data on Methods Bites</title>
    <link>https://socialsciencedatalab.mzes.uni-mannheim.de/tags/survey-data/</link>
    <description>Recent content in Survey data on Methods Bites</description>
    <generator>Hugo -- gohugo.io</generator>
    <lastBuildDate>Wed, 25 Mar 2026 00:00:00 +0000</lastBuildDate>
    
        <atom:link href="https://socialsciencedatalab.mzes.uni-mannheim.de/tags/survey-data/index.xml" rel="self" type="application/rss+xml" />
    
    
    <item>
      <title>Survey Recruitment on LinkedIn: A Step-by-Step Guide to Targeted Outreach</title>
      <link>https://socialsciencedatalab.mzes.uni-mannheim.de/article/linked-in/</link>
      <pubDate>Wed, 25 Mar 2026 00:00:00 +0000</pubDate>
      
      <guid>https://socialsciencedatalab.mzes.uni-mannheim.de/article/linked-in/</guid>
      <description><![CDATA[
        </p>
<!-- Optional: One paragraph on learning objectives -->
<p>LinkedIn’s professional user base and precise targeting capabilities present social science researchers with a valuable opportunity to recruit survey participants from highly specific populations, particularly those relevant to research on employment and industry dynamics. While platforms such as Facebook and Instagram are commonly used for recruitment of survey participants, LinkedIn’s potential in survey participant recruitment remains underutilized. Its professional focus, career-centric context, and only minimal off-topic content make it ideal for reaching the potential labor force population. This is particularly relevant for studies requiring insights from specific industries, occupations, or education levels.
In this Methods Bites Tutorial <a href="https://www.linkedin.com/in/zaza-zindel">Dr. Zaza Zindel</a> (German Centre for Integration and Migration Research (DeZIM)) and <a href="https://www.linkedin.com/in/lisa-de-vries-38465423a">Dr. Lisa de Vries</a> (FernUniversität in Hagen) provide a step-by-step guide on how to use LinkedIn ads to recruit participants for survey research.</p>
<p>After reading this blog post, readers will be able to:</p>
<ul>
<li>understand how LinkedIn advertisements can be used for survey participant recruitment,</li>
<li>set up targeted LinkedIn advertising campaigns, and</li>
<li>evaluate campaign performance and adapt recruitment strategies accordingly.</li>
</ul>
<!-- If applicable: Note with references on related materials -->
<!-- 
Generate an overview of the article 
Note: Section anchors are generated automatically from section headings as 
      hyphenated lower-case labels without special characters; e.g. a section
      title "Why R?" will result in the anchor "#why-r".
-->
<div id="overview" class="section level3">
<h3>Overview</h3>
<ol style="list-style-type: decimal">
<li><a href="#first-section"><strong>Introduction to LinkedIn ads as survey recruitment method</strong></a></li>
<li><a href="#second-section"><strong>Recruiting via LinkedIn – a step-by-step guide</strong></a>
<ol style="list-style-type: decimal">
<li><a href="#second-first-subsection">LinkedIn Campaign Manager</a>
<ol style="list-style-type: decimal">
<li><a href="#second-first-first-subsubsection">Personal LinkedIn account</a></li>
<li><a href="#second-first-second-subsubsection">LinkedIn ad account</a></li>
<li><a href="#second-first-third-subsubsection">LinkedIn page</a></li>
</ol></li>
<li><a href="#second-second-subsection">Setting up an ad campaign</a>
<ol style="list-style-type: decimal">
<li><a href="#second-second-first-subsubsection">Campaign level</a></li>
<li><a href="#second-second-second-subsubsection">Ad set level</a></li>
<li><a href="#second-second-third-subsubsection">Ad level</a></li>
</ol></li>
</ol></li>
<li><a href="#third-section"><strong>Ethical considerations</strong></a></li>
<li><a href="#fourth-section"><strong>Application Example</strong></a>
<ol style="list-style-type: decimal">
<li><a href="#fourth-first-subsection">Ad campaign details</a>
<ol style="list-style-type: decimal">
<li><a href="#fourth-first-first-subsubsection">Ad creatives</a></li>
<li><a href="#fourth-first-second-subsubsection">Campaign setup</a></li>
</ol></li>
<li><a href="#fourth-second-subsection">Campaign results</a></li>
<li><a href="#fourth-third-subsection">Sample composition</a></li>
</ol></li>
<li><a href="#fifth-section"><strong>Summary and key takeaways</strong></a></li>
<li><a href="#further-reading"><strong>Further reading</strong></a></li>
</ol>
</div>
<div id="first-section" class="section level3">
<h3>Introduction to LinkedIn ads as survey recruitment method</h3>
<p>Social media platforms have become an increasingly common tool for survey participant recruitment, allowing researchers to reach large populations quickly, cost-effectively, and with fine-grained targeting options. Most existing methodological work in this area, however, has focused on general-purpose platforms such as Facebook and Instagram. Systematic reviews of social media recruitment consistently show that these platforms dominate the field, whereas LinkedIn appears only sporadically and is rarely the central focus of empirical or methodological studies (e.g., <a href="https://doi.org/10.12758/mda.2022.15">Zindel 2023</a>; <a href="https://doi.org/10.7748/nr.2023.e1859">Jones et al. 2023</a>). This imbalance is striking, because LinkedIn offers a professionalized user base, rich profile information, and targeting parameters that are directly relevant to research on labor market and workplace related topics.</p>
<p>LinkedIn is a professional networking platform where people connect, share content, and participate in industry-specific communities. With more than one billion registered members worldwide (<a href="https://about.linkedin.com/">LinkedIn 2025</a>), it has become a central hub for working professionals, job seekers, and industry leaders. Because LinkedIn is explicitly work-oriented – and because user profiles contain rich educational and occupational information – it offers considerable potential for social science research. The platform mirrors real-world professional networks and career trajectories, making it particularly interesting for studies on labor market behavior, employment trends, and professional attitudes. A central strength of LinkedIn is the ability to reach narrowly defined populations based on industry, occupation, seniority, education, or geographic region. These segments are often difficult or impossible to access through traditional recruitment channels or general social media platforms. The granularity of profile information and targeting options enables researchers to sample very specific occupational groups or sectors, which is valuable for both descriptive and analytical research on employment and organizations. But next to the numerous possibilities of survey recruitment on LinkedIn, researchers must evaluate potential challenges regarding data protection, representativeness, and further potential bias.</p>
<p>To date, researchers use LinkedIn mostly either as an object of study – for example, to examine recruitment practices in labor markets, employer branding, or self-presentation strategies (e.g., <a href="https://doi.org/10.5958/2321-5763.2020.00010.4">Hosain &amp; Liu 2021</a>; <a href="https://doi.org/10.1177/0894439310386567">Caers &amp; Castelyns 2010</a>) – or as a data source for analyzing profile characteristics and network structures (e.g., <a href="https://doi.org/10.1016/j.chb.2018.08.033">Banerji &amp; Reimer 2019</a>; <a href="https://doi.org/10.1108/ER-07-2013-0086">Zide et al. 2014</a>). Some more recent work has begun to use LinkedIn to complement survey data, for instance by linking respondents’ answers to information from their LinkedIn profiles or from organizational pages (e.g., <a href="https://doi.org/10.1093/jssam/smae029">Al Baghal et al. 2024</a>). These studies illustrate that LinkedIn can provide rich contextual information on careers and organizations, but they typically focus on the use of existing profile data rather than on recruitment processes.</p>
<p>Researchers have also started experimenting with LinkedIn as a direct recruitment tool for survey-based studies. Documented approaches include promoting surveys through public (e.g., <a href="https://doi.org/10.1177/0193945917740706">Stokes et al. 2017</a>; <a href="https://doi.org/10.1093/swr/svaf002">Keemink et al. 2025</a>) or direct messages and personalized outreach (<a href="https://doi.org/10.1108/QRJ-04-2024-0085">Griffiths et al. 2025</a>; <a href="https://doi.org/10.1016/j.mex.2021.101393">Kozłowski et al. 2021</a>). However, methodological guidance on how to employ LinkedIn in a systematic and reproducible way remains limited. In particular, paid LinkedIn advertisements are still rarely documented in the academic literature. Existing accounts tend to be reflective case studies rather than detailed tutorials, and they often provide only high-level descriptions of campaign configurations and outcomes (e.g., <a href="https://doi.org/10.1186/s12889-023-16852-9">Kohl et al., 2023</a>).</p>
</div>
<div id="second-section" class="section level3">
<h3>Recruiting via LinkedIn – a step-by-step guide</h3>
<p>This section outlines a practical workflow for designing and implementing LinkedIn advertising campaigns for survey recruitment.</p>
<div id="second-first-subsection" class="section level5">
<h5>LinkedIn campaign manager</h5>
<p>Before launching any advertisements, several prerequisites must be met. Researchers require:</p>
<ol style="list-style-type: decimal">
<li>a personal LinkedIn account,</li>
<li>a LinkedIn ad account, and</li>
<li>a LinkedIn page.</li>
</ol>
<p>Each component serves a distinct purpose in the LinkedIn advertising ecosystem. The creation of all three components is free of charge.</p>
<div id="second-first-first-subsubsection" class="section level6">
<h6>Personal LinkedIn account</h6>
<p>Advertising on LinkedIn requires an active personal account associated with a real individual (i.e., a verifiable professional identity). During registration, the user must provide standard information such as name, location, job title, and current employer. For authenticity and compliance with LinkedIn’s advertising policies, the account should not be newly created: it must be at least 24 hours old, have at least one verified connection, and represent a verifiable professional identity. These criteria ensure that advertisements are traceable to legitimate users rather than anonymous entities or automated accounts. Importantly, these personal accounts are used for access and accountability within LinkedIn’s advertising system, but they are not displayed as the public-facing identity of the ads. Instead, the advertisements are shown as being placed by the LinkedIn Page (described below), meaning it is not externally visible which individual account initiated or manages a given campaign.</p>
</div>
<div id="second-first-second-subsubsection" class="section level6">
<h6>LinkedIn ad account</h6>
<p>Once a personal profile exists, an advertising account can be established via the <a href="https://business.linkedin.com/advertise">LinkedIn Advertise platform</a>. Selecting “Create ad” or “Get Started” initiates the process (see Figure <a href="#fig1">1</a>).</p>
<a name="fig1"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig1"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInMarketingSolution.png" alt="Starting page of LinkedIn Advertise website." width="70%" />
<p class="caption">
Figure 1: Starting page of LinkedIn Advertise website.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://business.linkedin.com/advertise">LinkedIn Advertise</a>
</sup></sub>
</p>
</div>
<p>The platform then prompts the user to assign an account name and link the account to a LinkedIn Page (see Figure <a href="#fig2">2</a>). If no appropriate institutional page is available, a new one must be created.</p>
<a name="fig2"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig2"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAdsAccount.png" alt="LinkedIn ads account set up." width="70%" />
<p class="caption">
Figure 2: LinkedIn ads account set up.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager/new-advertiser">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>After this step, the ad account provides access to campaign creation, audience definition, and budgeting functions.</p>
</div>
<div id="second-first-third-subsubsection" class="section level6">
<h6>LinkedIn page</h6>
<p>A LinkedIn page functions as the institutional or organizational identity under which advertisements are displayed. Depending on the project’s context, the page may represent:</p>
<ul>
<li>a company or organization (e.g., a research institute),</li>
<li>a showcase page (a subpage linked to a main institutional account), or</li>
<li>an educational institution (e.g., an University).</li>
</ul>
<p>Unlike platforms such as Facebook (Meta), LinkedIn does not currently offer a dedicated project or educational page type, meaning researchers must register under one of the existing categories. Each page requires basic information such as a name, a public URL, the industry, and the size of the organization (see Figure <a href="#fig3">3</a>). In addition, it is possible to indicate further information (i.e., logo and a short organizational slogan). For research recruitment, using an institutional page (for example, a university department or research group) can increase credibility and transparency in the eyes of potential participants.</p>
<a name="fig3"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig3"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInPage.png" alt="Set up LinkedIn page – example: educational institute." width="60%" />
<p class="caption">
Figure 3: Set up LinkedIn page – example: educational institute.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager/new-advertiser">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
</div>
</div>
<div id="second-second-subsection" class="section level5">
<h5>Setting up an ad campaign</h5>
<p>The LinkedIn advertising system follows a three-tier hierarchical structure:</p>
<a name="fig4"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig4"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAdvertisingSystem.png" alt="LinkedIn advertising system" width="80%" />
<p class="caption">
Figure 4: LinkedIn advertising system
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: Own illustration.
</sup></sub>
</p>
</div>
<p>Each tier governs a different aspect of the set-up, from defining overall objectives to specifying detailed audience characteristics and ad creatives. Understanding this structure is essential for organizing an effective and transparent recruitment effort.</p>
<div id="second-second-first-subsubsection" class="section level6">
<h6>Campaign level</h6>
<p>At the highest level, the campaign establishes the overarching goal and organizational framework of the recruitment project. LinkedIn offers three types of objectives (see Figure <a href="#fig5">5</a>):</p>
<ul>
<li>Awareness (increasing visibility),</li>
<li>Consideration (encouraging engagement), and</li>
<li>Conversion (prompting specific actions).</li>
</ul>
<p>For survey recruitment, the “Website visits” option under the Consideration category is generally most suitable, as it directs users to an external landing page, typically the survey entry point.</p>
<a name="fig5"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig5"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInObjective.png" alt="Choosing a campaign objective" width="60%" />
<p class="caption">
Figure 5: Choosing a campaign objective
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>Once the campaign objective has been chosen, LinkedIn prompts the user to select a campaign type. Two options are available: a manual setup (“Classic”) and an AI-assisted setup (“Accelerate”) (see Figure <a href="#fig6">6</a>). While the latter automatically adjusts parameters to optimize engagement, we strongly recommend avoiding it. Algorithmic optimization introduces unknown and uncontrolled sources of bias into audience selection, thereby comprising the transparency and replicability required in scientific research. Manual configuration ensures that all targeting and budgeting decisions remain under the advertiser’s, that is, the researcher’s control and can be documented in detail.</p>
<a name="fig6"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig6"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInCampaignType.png" alt="Selecting a campaign type" width="70%" />
<p class="caption">
Figure 6: Selecting a campaign type
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>After selecting the campaign type, researchers must determine the ad creation process, that is, how individual advertisement will be assembled and optimized within the campaign. Again, LinkedIn offers two routes: Flexible ad creation and standard ad creation (see Figure <a href="#fig7">7</a>).
In the flexible ad creation mode, the platform automatically generates and tests different combinations of ad elements (e.g., images, headlines, and descriptions) and allocates the budget to those combinations that achieve the best predicted performance. This process resembles A/B testing but is fully automated. Although potentially efficient, this approach lacks transparency. The platform does not disclose the criteria used to define “best performance” nor how these predictions are generated. Consequently, it introduces algorithmic bias and variation that are not under research control.
In research contexts, the standard ad creation mode is clearly preferable. This mode requires researchers to manually configure each ad creatively. All text and visual elements, as well as audience and delivery settings, are explicitly defined by the researchers. This manual process ensures that recruitment materials remain consistent and that all campaign parameters are fully reproducible and can be reported in a methods section.</p>
<a name="fig7"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig7"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInHowCreate.png" alt="Selecting an ad creation process" width="70%" />
<p class="caption">
Figure 7: Selecting an ad creation process
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>After determining the creation route, the campaign configuration also includes the definition of schedule and budget parameters (see Figure <a href="#fig8">8</a>). Researchers may specify:</p>
<ul>
<li>a start and end date for the campaign,</li>
<li>a total (lifetime) budget, and</li>
<li>whether budget optimization across campaigns should be enabled.</li>
</ul>
<p>Again, we recommend disabling all forms of automated budget optimization. Although these features can improve marketing efficiency by reallocating funds to higher-performing ads, they rely on algorithmic assessments that are not accessible to researchers and could bias the recruited sample in unknown and unobservable ways.</p>
<a name="fig8"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig8"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInGroupBudget.png" alt="Campaign details" width="70%" />
<p class="caption">
Figure 8: Campaign details
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
</div>
<div id="second-second-second-subsubsection" class="section level6">
<h6>Ad set level</h6>
<p>At ad set level, the researcher defines the target population, that is, the group of LinkedIn users who will be exposed to the survey advertisement. LinkedIn provides numerous targeting attributes derived from the members’ professional profiles, company pages, and inferred behavioral or interest signals (see Figure <a href="#fig9">9</a>). Table <a href="#table1">1</a> provides a comprehensive overview of the main attribute categories and typical options within each. Where known, information on how LinkedIn obtains or infers that information is indicated.</p>
<center>
<a name="table1"></a>
<caption>
Table 1: LinkedIn Attributes and Data Sources
</caption>
<iframe src="/article/linked-in/Table1.htm" width="100%" height="840px" data-external="1">
</iframe>
<div style="text-align: centre">
<p>
<sub><sup>
Note: Data availability and functionality may vary by region due to privacy regulations, particularly with the Europeam Economic Area and Switzerland.
</sup></sub>
</p>
</div>
</center>
<a name="fig9"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig9"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAudience.png" alt="Definition of target audience" width="70%" />
<p class="caption">
Figure 9: Definition of target audience
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>As seen in Table <a href="#table1">1</a>, researchers have various options for target parameters. Therefore, is important that targeting parameters must never be used to discriminate against individuals based on protected characteristics such as gender, age, or actual or perceived race or ethnicity. Researchers should also comply with platform-specific restrictions and regional regulations that limit the use of sensitive targeting criteria.
The option “enable audience expansion” should be deactivated (see Figure <a href="#fig9">9</a>). This algorithmic extension may deliver ads to users outside the intended target population and thus undermine the transparency and controllability of the sample frame.
LinkedIn’s Campaign Manager also displays an estimated performance forecast, showing projected target audience size, impressions, clicks, and costs (see Figure <a href="#fig10">10</a>). These values can assist in budget planning but are based on proprietary models and should be interpreted as approximate indicators only, not as precise predictions.</p>
<a name="fig10"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig10"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInForecast.png" alt="Forecasted results" width="25%" />
<p class="caption">
Figure 10: Forecasted results
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>Once the campaign objectives and audience have been defined, the next step is the selection of an appropriate advertisement format (see Figure <a href="#fig11">11</a>). LinkedIn currently offers several formats that differ in visual layout, degree of interactivity, and expected engagement levels. The choice of format should align with the overall research design, recruitment objectives, and desired intensity of participation involvement. Available formats include, among other:</p>
<ul>
<li>Single image ads: static image with headline and text,</li>
<li>Carousel ads: multiple scrollable image cards that allow the presentation of different aspects or topics within one advertisement, and</li>
<li>Video ads: short clip (15-30 seconds) combining visuals and text to communicate information dynamically.</li>
</ul>
<p>For most survey recruitment, these formats are generally the most suitable due to their high visibility and clear message presentation.</p>
<a name="fig11"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig11"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAdFormat.png" alt="Ad format" width="65%" />
<p class="caption">
Figure 11: Ad format
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>The integration of URL tracking parameters (e.g., UTM codes) into the destination link is recommended to enable transparent performance monitoring (see Figure <a href="#fig12">12</a>). A typical example would be:</p>
<pre class="text"><code>utm_source=[SOCIAL MEDIA PLATFORM]&amp;utm_medium=[PAYMENT EVENT]
&amp;utm_campaign=[CAMPAIGN NAME]&amp;utm_content=[AD ID].</code></pre>
<p>This method complies with data protection requirements and does not collect personal information. It simply allows researchers to distinguish traffic from different platforms or creatives in their web analytics.</p>
<a name="fig12"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig12"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInURL.png" alt="URL tracking parameters" width="65%" />
<p class="caption">
Figure 12: URL tracking parameters
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>Ad placement is then specified (see Figure <a href="#fig13">13</a>). The LinkedIn feed should be selected as the primary placement to ensure visibility in a professional context. The LinkedIn Audience Network, which extends ad delivery to partner websites and apps, should be deactivated in research campaigns to maintain control and transparency regarding where exactly recruitment occurs.</p>
<a name="fig13"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig13"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInPlacements.png" alt="Placements option" width="65%" />
<p class="caption">
Figure 13: Placements option
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>The budgeting model determines how funds are distributed over time (Figure <a href="#fig14">14</a>). LinkedIn provides three options:</p>
<ul>
<li>Daily budget: specifies maximum daily expenditure, ensuring consistent ad delivery,</li>
<li>Lifetime budget: sets a total expenditure limit for the campaign period, and</li>
<li>Combined budget: applies both daily and lifetime limits for maximum control.</li>
</ul>
<p>Short-term campaigns benefit from daily budgets to maintain stable exposure, while longer recruitment periods may require lifetime or combined budget settings. Key performance metrics such as impressions, click-through rate (CTR), and cost per click (CPC) should be monitored throughout the campaign and reported alongside sample characteristics.</p>
<a name="fig14"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig14"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInBudgetSchedule.png" alt="Budget and schedule options" width="60%" />
<p class="caption">
Figure 14: Budget and schedule options
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>The bidding strategy determines how LinkedIn allocates ads in its auctions system (Figure <a href="#fig15">15</a>). LinkedIn currently offers three bid types:</p>
<ul>
<li>Maximum delivery: an automated bidding option where LinkedIn’s system sets the bid with the goal of spending the full budget and maximizing the selected key result,</li>
<li>Cost cap: an automated option where LinkedIn sets bids while trying to keep the average cost per key result close to a user-specified target (e.g., CPC or CPM), and</li>
<li>Manual bidding: a more hands-on option where advertisers set an explicit bid value that is used in the ad auction.</li>
</ul>
<p>From a research perspective, there is a trade-off between cost-efficiency and methodological control. Fully automated options such as maximum delivery can be convenient and may help use the budget efficiently, but they also shift additional decision-making to opaque algorithms. Where fine-grained cost control and transparency are priorities, cost cap or even manual bidding may be preferable, provided that campaigns are monitored closely, and bid levels are adjusted if necessary. Regardless of the chosen strategy, researchers should document the bid type and any changes during fieldwork so that recruitment conditions can be accurately reported.</p>
<a name="fig15"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig15"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInBidding.png" alt="Bidding options" width="70%" />
<p class="caption">
Figure 15: Bidding options
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>Although LinkedIn supports conversion tracking via tracking pixels (see Figure <a href="#fig16">16</a>), this method is ethically and legally problematic in research contexts. Tracking pixels typically collect behavioral data across websites (e.g., page views, time spent, subsequent actions) and can be used to link on-platform interactions with off-platform behavior. Under data protection frameworks such as the General Data Protection Regulation (GDPR), such tracking requires a valid legal basis and, in most cases, informed consent: participants must be clearly informed before any tracking takes place about what data are collected, for what purpose, on which legal basis, how long they are stored, and with whom they may be shared. Only if users have actively agreed (e.g., via a consent banner or consent form on the landing page) may tracking pixels be activated.
However, for scientific campaigns, this form of tracking is usually difficult to justify. Social media platforms and advertising networks have repeatedly been affected by data breaches and other security incidents (e.g., <a href="https://www.theguardian.com/uk-news/2023/jul/15/revealed-metropolitan-police-shared-sensitive-data-about-victims-with-facebook">Das 2023</a>; <a href="https://themarkup.org/pixel-hunt/2022/06/16/facebook-is-receiving-sensitive-medical-information-from-hospital-websites">Feathers et al. 2022</a>; <a href="https://themarkup.org/pixel-hunt/2022/11/22/tax-filing-websites-have-been-sending-users-financial-information-to-facebook">Fondrie-Teitler et al. 2022</a>). When third-party tracking is enabled, additional actors gain access to behavioral data and identifiers, increasing the potential attack surface. Even if the probability of a breach is difficult to quantify, the possible consequences for participants – for example, the unintended disclosure of sensitive behavioral patterns or re-identification risks when log data are leaked – can be substantial. From a risk-minimization perspective, avoiding non-essential tracking is therefore a central protective measure.
We therefore recommend that researchers do not use LinkedIn’s conversion tracking or similar third-party tracking pixels for recruitment. Instead, evaluation should rely on aggregate campaign statistics (impressions, clicks, CPC) provided by the platform, combined with survey-based indicators (e.g., completion rates and a self-reported recruitment channel question on the first page of the questionnaire, asking participants how they became aware of the study).</p>
<a name="fig16"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig16"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInConversion.png" alt="Conversion tracking" width="80%" />
<p class="caption">
Figure 16: Conversion tracking
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
</div>
<div id="second-second-third-subsubsection" class="section level6">
<h6>Ad level</h6>
<p>At the ad level, the creative elements that LinkedIn users are exposed to are defined.
The design of these elements can draw on established findings from research on invitation letters in survey methodology. In line with this literature, ad content should be convincing and informative, and should convey a clear perceived leverage for participation, for example by highlighting the social value of the study, the relevance of the topic for the target population, or the opportunity to contribute one’s perspective (e.g., Dillman et al. 2014 ; <a href="http://www.jstor.org/stable/3078721">Groves et al. 2000</a>). On LinkedIn, as on other social media platforms, an ad competes with a large volume of other digital content. Consequently, the ad must first attract attention within a very short time frame and then provide sufficient relevance and clarity to motivate users to click on the ad and proceed to the survey landing page.
The introductory text (Figure <a href="#fig17">17</a>) – typically the most prominent text element in the ad – should succinctly communicate the purpose of the study and its relevance for the targeted audience. A concise structure is advisable: one sentence specifying what the study is about and who it addresses, followed by a sentence that conveys why participation is meaningful (e.g., expected benefits, contribution to knowledge, or policy relevance).</p>
<a name="fig17"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig17"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAdDesign.png" alt="Ad design  - Text and media input" width="80%" />
<p class="caption">
Figure 17: Ad design - Text and media input
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>The visual content serves as the primary attention grabber and can function as an additional targeting device (Figure <a href="#fig17">17</a>). On LinkedIn, visuals are often embedded in a professional and organizational context, for example by depicting typical work situations (e.g., team meetings, hybrid collaboration, industry-specific workplaces), abstract graphics related to the topic (e.g., charts, icons for digitalization or skills), or neutral employer-branding style imagery. When these visuals are clearly aligned with a specific survey topic – for instance, leadership and management practices, remote work arrangements, skills development, or employee well-being – they can increase click-through rates and the number of completed questionnaires, because they immediately signal topical relevance to certain occupational groups or sectors. At the same time, these visuals may amplify self-selection on work-related attitudes or experiences (e.g., job satisfaction, perceived stress, views on organizational change), as they are particularly salient for individuals with above-average involvement or strong opinions (<a href="https://doi.org/10.12758/mda.2025.05">Donzowa et al. 2025</a>; <a href="https://doi.org/10.1080/13645579.2025.2597303">Zindel et al., 2025</a>). This implies a trade-off: if recruitment within a narrowly defined topic area is prioritized, topic-specific professional visuals are advantageous; if concerns about attitudinal bias are central, more neutral or general workplace imagery may be preferable.
In addition to the visual itself, LinkedIn offers the option to set an image alt text (Figure <a href="#fig18">18</a>). This alt text enhances accessibility for users who rely on screen readers and is displayed if the image cannot be rendered. From a survey recruitment perspective, the alt text should provide a concise, neutral description of what is shown in the image and, where appropriate, can briefly restate the topic of the study (e.g., “Illustration of professionals discussing climate policies in an office setting”). Overly promotional language or keyword stuffing should be avoided. The goal is to ensure that the meaning of the visual remains understandable even in the absence of the actual image and that the alt text is consistent with the overall framing of the study (cf. [World Wide Web Consortium [W3C], 2025[(<a href="https://www.w3.org/TR/WCAG21/" class="uri">https://www.w3.org/TR/WCAG21/</a>)]).</p>
<a name="fig18"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig18"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAdDesignOption.png" alt="Ad design - Further options" width="85%" />
<p class="caption">
Figure 18: Ad design - Further options
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager</a>
</sup></sub>
</p>
</div>
<p>The destination URL is the link to the survey website and leads directly to the survey entry page, without intermediary pages or redirects. This entry page must present the informed consent statement and essential information on data protection, study sponsors, and contact details before data collection begins. From a user-experience perspective, the number of steps between the ad (i.e., the survey invitation) and the first survey question should be minimized, as each additional step potentially increases drop-off probabilities (cf. <a href="https://doi.org/10.1093/acprof:oso/9780199747047.001.0001">Tourangeau et al. 2013</a>).
Besides the introductory text and the image, LinkedIn ads typically include a headline and an optional ad description. The headline is usually displayed in a highly prominent position and should therefore be short, specific, and benefit oriented. For survey recruitment, headlines that clearly state the topic and target group (e.g., “Short survey on remote work for HR professionals”) are generally preferable to vague or purely promotional formulations.
The ad description provides additional information to people who see the ad. However, this field is not visible in most scenarios and will only appear for a small portion of LinkedIn members, for example in certain placements on the LinkedIn Audience Network (if enabled). Accordingly, essential information about the study should not be placed exclusively in the description but should already be contained in the introductory text and headline. The description can be used as a secondary field to elaborate on the headline, clarify what happens after the click, indicate approximate survey length, or highlight a specific leverage (e.g., “10-minute academic survey, anonymous, results will inform future workplace policies”). Given that descriptions may be truncated or omitted depending on device and placement, key information should appear at the beginning of the text and should not be critical for understanding the study.
The call-to-action (CTA) button should be consistent with the overall framing of the ad and align with user expectations on a professional platform. LinkedIn offers several standard CTA options, including “Learn more”, “Sign up”, “Register”, and others. For survey recruitment, a CTA such as “Learn more” is typically most appropriate, as it signals low-threshold, information-oriented action compatible with participation in a research study. In settings where the survey is embedded in a broader program or panel, “Sign up” or “Register” may also be suitable.</p>
<a name="fig19"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig19"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAdPreview.png" alt="Ad preview" width="50%" />
<p class="caption">
Figure 19: Ad preview
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager.</a> Image by Africa Studio via <a href=“https.//stock.adobe.com/115404199“>stock.adobe.com.</a>
</sup></sub>
</p>
</div>
<p>Prior to launch, the preview function in Campaign Manager should be used to verify appearance across devices and placements (see Figure <a href="#fig19">19</a>). Across all ad components, internal consistency is crucial. The introductory text, headline, description, visual elements (including alt text), destination URL, and CTA should convey a coherent message and set realistic expectations about the survey experience. A consistent, methodologically informed ad design that is at the same time adapted to platform-specific norms on LinkedIn potentially increases the likelihood of attracting suitable respondents rather than merely generating clicks.</p>
</div>
</div>
</div>
<div id="third-section" class="section level3">
<h3>Ethical considerations</h3>
<p>When recruiting survey participants via social media, particular attention should be paid to ethical and data protection issues. Most important, all procedures must comply with applicable data protection frameworks (e.g., GDPR), including a clear specification of the legal basis for processing, information on storage periods, and contact details of the responsible data controller and, where applicable, the data protection officer. Moreover, recruitment messages and landing pages should be transparent about the study’s purpose, the institutions involved, and the intended use of the data. Participants should be informed about their data protection rights (e.g., rights of access, rectification, erasure, restriction of processing, and complaint) in a concise and comprehensible way.
Where surveys are anonymous, this should be explicitly communicated and technically ensured, for instance by avoiding the collection of directly identifying information and by separating contact data from survey responses if recontact is planned. Where anonymity cannot be fully guaranteed – for example because organization-level or very narrow professional targeting makes the population of potential invitees very small – this limitation should be stated clearly. In such cases, it should be emphasized that confidentiality will nonetheless be preserved, for example through secure data handling and the exclusive reporting of results in aggregated form.
Given that recruitment on LinkedIn and similar platforms can take place at very specific levels (e.g., by employer, job title, industry niche, or narrowly defined professional segments), the risk of de facto identifiability of respondents may be higher than in general-population web surveys. This applies in particular when small organizations or rare occupations are involved. Ethical scrutiny is therefore essential. Targeting strategies should avoid extremely small and potentially identifiable groups where this is not strictly necessary for the research aim, and analysis and reporting should follow minimum cell-size rules to prevent the indirect identification of individuals or specific organizations.
It is strongly recommended to seek approval or at least formal advice from the relevant ethics committee at the home institution or from an external ethics body before fieldwork begins. An ethics review helps to ensure that the sampling and recruitment strategy is appropriate, that privacy and confidentiality risks are minimized, that the informed consent procedure is adequate, and that the study aligns with existing ethical guidelines and professional codes of conduct.</p>
</div>
<div id="fourth-section" class="section level3">
<h3>Application example</h3>
<p>This section illustrates how the LinkedIn recruitment workflow described above can be applied in practice. It summarizes the design, implementation, and outcomes of an exemplary advertising campaign conducted as part of a master’s seminar at Bielefeld University in summer term 2024 titled “Sociological methods - quantitative: Survey-based measurement of inclusion and diversity in companies: Development and implementation of employee surveys”, taught by Zaza Zindel and Lisa de Vries.<br />
Within this seminar, students developed a questionnaire to measure inclusion and diversity in workplace settings. The instrument comprised 37 items, including the following topics:</p>
<ul>
<li>Working life (7 items)</li>
<li>Career orientation (3 items)</li>
<li>Diversity (3 items)</li>
<li>Discrimination (4 items)</li>
<li>Health (6 items)</li>
<li>Sociodemographic (13 items)</li>
<li>Closing question (1 item).</li>
</ul>
<p>The target population consisted of employees in Germany aged 18 years or older. The survey was programmed using the software LimeSurvey. To reach the target population without cooperation from specific companies, survey recruitment was carried out via LinkedIn ads. A total budget of 500 Euro was allocated for the campaign.</p>
<div id="fourth-first-subsection" class="section level4">
<h4>Ad campaign details</h4>
<p>Following the three-tier campaign structure described above, the campaign was configured at the campaign, ad set, and ad levels. A dedicated LinkedIn page titled “Empirische Sozialforschung – Uni Bielefeld” (English translation: “Empirical Social Research – Bielefeld University” was created to host the ads and serve as the institutional identity for the recruitment process (see Figure <a href="#fig20">20</a>).</p>
<a name="fig20"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig20"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInEmpiricalPage.png" alt="LinkedIn page: Empirical social research – Bielefeld University" width="80%" />
<p class="caption">
Figure 20: LinkedIn page: Empirical social research – Bielefeld University
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager.</a> Images by <a href=“https.//stock.adobe.com/115404199“>Monster Ztudio</a> and <a href=“https.//stock.adobe.com/390374250“>Siberian Art</a> via stock.adobe.com.
</sup></sub>
</p>
</div>
<div id="fourth-first-first-subsubsection" class="section level6">
<h6>Ad creatives</h6>
<p>Two separate advertisements in the single image format were developed. They used identical textual content but different ad images to examine how visuals influence engagement (Figures <a href="#fig21">21</a> and <a href="#fig22">22</a>).</p>
<p><strong>Headline:</strong> We want your opinion: Take part in our survey on the German labor market!
<strong>Introductory text:</strong> Help us better understand the German labor market! We are looking for participants for a short survey that is part of a scientific study conducted by Bielefeld University. The survey is aimed at all employed people aged 18 and over in Germany.</p>
<a name="fig21"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig21"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAd1.png" alt="Targeting ad 1 used to recruit participants on LinkedIn" width="60%" />
<p class="caption">
Figure 21: Targeting ad 1 used to recruit participants on LinkedIn
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager.</a> Image by mitay20 via <a href=“https.//stock.adobe.com/307640218“>stock.adobe.com</a>.
</sup></sub>
</p>
</div>
<a name="fig22"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig22"></span>
<img src="../../../../../../../../article/linked-in/images/LinkedInAd2.png" alt="Targeting ad 2 used to recruit participants on LinkedIn" width="60%" />
<p class="caption">
Figure 22: Targeting ad 2 used to recruit participants on LinkedIn
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.linkedin.com/campaignmanager">LinkedIn Campaign Manager.</a> Image by Africa Studio via <a href=“https.//stock.adobe.com/115404199“>stock.adobe.com.</a>
</sup></sub>
</p>
</div>
</div>
<div id="fourth-first-second-subsubsection" class="section level6">
<h6>Campaign setup</h6>
<p>Both advertisements employed identical targeting criteria to ensure comparability across creatives. To recruit broadly from employed individuals in Germany, targeting was restricted to:</p>
<ul>
<li>Location: Germany</li>
<li>Profile Language: German</li>
<li>Age: 18+ years</li>
</ul>
<p>No additional targeting criteria were applied, and audience expansion was disabled. The placement was set to LinkedIn Feed only and audience network placements was disabled. A lifetime budget of 500 EUR was defined, optimization goal was landing page clicks, and the bidding strategy was maximum delivery until all the budget is spent.</p>
</div>
</div>
<div id="fourth-second-subsection" class="section level4">
<h4>Campaign results</h4>
<p>The LinkedIn recruitment campaign ran for five consecutive days, from 6 to 10 July 2024, and was managed through LinkedIn Campaign Manager using the institutional page created for this project.
Across both advertisements, the campaign reached a substantial professional audience and generated a sizeable number of completed interviews within a modest budget. Table <a href="#table2">2</a> summarizes the performance metrics for both advertisements. In total, the two ads produced 56,990 impressions, resulting in 301 link clicks. Of those who clicked on the advertisement, 143 started the interview and 98 completed the questionnaire. With a total spend of 500 EUR, this corresponds to an overall cost per click (CPC) of 1.66 EUR and a cost per completed interview (CI) of 5.10 EUR. These results suggest that LinkedIn Ads can serve as an efficient recruitment channel for workplace-related surveys, producing a relatively high number of completed questionnaires at moderate cost.</p>
<center>
<a name="table2"></a>
<caption>
Table 2: Results of ad and survey performance
</caption>
<br>
<iframe src="/article/linked-in/Table2.htm" width="70%" height="270px" data-external="1"></iframe>
<div style="text-align: centre">
<p>
<sub><sup>
Note: CTR = Click-through-Rate, CPC = Cost per click, CI = completed interview.
</sup></sub>
</p>
</div>
</center>
<p>Both ads achieved a click-through rate (CTR) of roughly 0.5% (0.54% for Ad 1 and 0.52% for Ad 2). The overall CPC was 1.66 EUR, with a slightly lower CPC for Ad 1 (1.58 EUR) than for Ad 2 (1.73 EUR). At the same time, Ad 2 produced more completed interviews (57 compared to 41) and therefore a slightly lower cost per completed interview (5.03 EUR for Ad 2 versus 5.20 EUR for Ad 1). These differences are numerically small, but they show that the two creatives did not behave identically in terms of cost structure, even though they were identical in text and targeting.
The delivery patterns of the two ads are also differed. Ad 1 was shown to a larger number of unique users (reach 7,269) and with a lower average frequency (3.45 displays per user). Ad 2, by contrast, reached fewer unique users (5,189) but was displayed more often to those who were reached (average frequency 6.15). Thus, the total number of impressions was higher for Ad 2, even though its reach was lower. Within the constraints of the shared targeting, this indicates that LinkedIn’s delivery system concentrated Ad 2 more strongly on a subset of the potential audience, while Ad 1 was distributed more broadly.</p>
</div>
<div id="fourth-third-subsection" class="section level4">
<h4>Sample composition</h4>
<p>The composition of the achieved sample reflects the characteristics of those LinkedIn users who clicked on the advertisements, proceeded to the survey, and completed the questionnaire. Table <a href="#table3">3</a> shows the distribution of the sociodemographic variables. Respondents are distributed across all age groups included in the questionnaire, with the largest shares in the ranges 45–54 years (31.6%) and 35–44 years (29.6%) (age mean: 45.84; Range: 23-69). Regarding gender, 62 respondents identify as female and 32 as male, while four respondents did not answer this question. No respondent selected the response option for another gender identity. Education attainment in the sample is strongly concentrated at the higher end of the scale: Most respondents have a higher secondary school diploma (85.7 %), further 10.2 percent hold a secondary school diploma.
If we compare the composition of sociodemographic variables with the whole population in Germany, we see that the sample contain relatively few young people (<a href="https://ergebnisse.zensus2022.de/datenbank/online/statistic/1000A/table/1000A-1017">Statistisches Bundesamt 2025a</a>). Even if 21.4 percent of the sample are 55 years and older, only 10.4 percent are 60 years and older, and no respondents are older than 70. In 2023, 29,8 percent of the population were 60 years and older based on Zensus (<a href="https://ergebnisse.zensus2022.de/datenbank/online/statistic/1000A/table/1000A-1017">Statistisches Bundesamt 2025a</a>). These differences may be due to the fact that LinkedIn is used particularly by people of working age who are neither in education nor retired. Moreover, the percentage of women and the educational attainment in the sample is higher than in the whole population in Germany (<a href="https://ergebnisse.zensus2022.de/datenbank/online/statistic/1000A/table/1000A-1017">Statistische Ämter des Bundes und der Länder 2025a</a>; <a href="https://www.destatis.de/DE/Themen/Arbeit/Arbeitsmarkt/Erwerbstaetigkeit/Tabellen/eckwerttabelle.html">Statistische Ämter des Bundes und der Länder 2025b</a>).</p>
<center>
<a name="table3"></a>
<caption>
Table 3: Sociodemographic variables
</caption>
<br>
<iframe src="/article/linked-in/Table3.htm" width="90%" height="430px" data-external="1"></iframe>
<div style="text-align: centre">
<p>
<sub><sup>
Source: Dataset “DEI Companies” recruited on LinkedIn; own calculations.
</sup></sub>
</p>
</div>
</center>
<p>Table <a href="#table4">4</a> presents the occupational variables. Most respondents report full-time employment (75.5%), while 19.4 percent work part-time. Only four respondents indicated marginal or no employment. In 2024 in Germany, the unemployment rate was 3.1 percent, and the part-time rate was 29,1 percent (<a href="https://www.destatis.de/DE/Themen/Arbeit/Arbeitsmarkt/Erwerbstaetigkeit/Tabellen/eckwerttabelle.html">Statistisches Bundesamt 2025b</a>; <a href="https://www.destatis.de/DE/Themen/Querschnitt/Gleichstellungsindikatoren/teilzeitquote-f25.html">Statistisches Bundesamt 2025c</a>). Firm size is also skewed toward larger organizations: 58.2 percent of the respondents work in companies with more than 200 employees, and 24.5 percent work in firms with between 20 and 200 employees. Only 15.3 percent of respondents report working in small firms with fewer than 20 employees. Based on Destatis, in 2023, nearly the half of the German population work in large companies (<a href="https://www.destatis.de/DE/Themen/Branchen-Unternehmen/Unternehmen/Kleine-Unternehmen-Mittlere-Unternehmen/aktuell-beschaeftigte.html">Statistisches Bundesamt 2025d</a>). With regard to leadership roles, 24.5 percent indicate that they hold a leadership position, while 59.2 percent do not. Notably, 16 respondents did not answer this question, resulting in a higher share of missing values than the other occupational variables. The distribution across industry sectors covers a broad range of fields. The largest groups of respondents work in manufacturing (15.3%), information and communication (15.3%), and human health and social work activities (15.3%). Other sectors represented in the sample include scientific and technical activities, education, financial and insurance activities, public administration, and several smaller categories. Even if the share of people working in the manufacturing sector is smaller in the sample than in the general population, that a relatively high percentage work in the area of services (e.g., education or human health and social work activities) is also true in the general population (<a href="https://www.destatis.de/DE/Themen/Wirtschaft/Konjunkturindikatoren/Lange-Reihen/Arbeitsmarkt/lrerw13a.html">Statistisches Bundesamt 2025e</a>; <a href="https://www.destatis.de/DE/Themen/Arbeit/Arbeitsmarkt/Erwerbstaetigkeit/Tabellen/arbeitnehmer-wirtschaftsbereiche.html#fussnote-2-122230">Statistisches Bundesamt 2025f</a>).</p>
<center>
<a name="table4"></a>
<caption>
Table 4: Occupational variables
</caption>
<br>
<iframe src="/article/linked-in/Table4.htm" width="65%" height="625px" data-external="1"></iframe>
<div style="text-align: centre">
<p>
<sub><sup>
Source: Dataset “DEI Companies” recruited on LinkedIn; own calculations.
</sup></sub>
</p>
</div>
</center>
<p>The discussed patterns are consistent with previous evidence that LinkedIn use is concentrated among higher-income professionals and knowledge-intensive occupations (<a href="https://doi.org/10.1177/0163443712468605">Van Dijck, 2013</a>; <a href="https://doi.org/10.1177/0002764217717559">Blank &amp; Lutz, 2017</a>) and underlines that LinkedIn-based samples capture a selective segment of the working-age population.</p>
</div>
</div>
<div id="fifth-section" class="section level3">
<h3>Summary and key takeaways</h3>
<p>LinkedIn provides a distinctive infrastructure for survey recruitment in professional and occupational populations. Its career-focused user base, comparatively low noise-to-signal ratio, and rich profile data on education, industry, and employment make it particularly suitable for studies on labor markets, workplace dynamics, and professional attitudes. In contrast to general-purpose platforms such as Instagram or TikTok, LinkedIn is designed around work, networking, and expertise. As a result, it tends to reach individuals of working age, often with medium to high levels of education and strong labor market attachment, which is reflected in the sample composition of the application example presented in this tutorial. This step-by-step guide provided an overview of how to use LinkedIn ads to recruit participants for survey research.
The application example from Bielefeld University illustrates that with a moderate budget, LinkedIn can yield a substantial number of completed interviews at reasonable cost, while also highlighting typical patterns such as the overrepresentation of highly educated respondents and specific age groups.</p>
<p>Taken together, the key takeaways from this tutorial can be summarized as follows:</p>
<ol style="list-style-type: decimal">
<li>Alternative recruitment channel: LinkedIn offers a viable alternative to conventional sampling approaches and general social media platforms for reaching employed and occupationally defined populations.</li>
<li>Distinct user structure: Compared to platforms such as Instagram or TikTok, LinkedIn tends to reach individuals in mid-career and higher educational strata, which is advantageous for many labor market-related studies.</li>
<li>Fine-grained targeting: Detailed targeting by job title, industry, company size, skills, and other professional attributes enables access to niche populations that are otherwise difficult to sample.</li>
<li>Selective coverage and bias: Only certain groups are reachable via LinkedIn, and topic-specific visuals or messaging can reinforce self-selection, requiring careful interpretation of substantive findings.</li>
<li>Heightened ethical responsibilities: Because targeting can be highly specific, ethical issues around privacy, identifiability, and fair treatment are particularly salient and call for robust consent procedures, conservative reporting practices, and prior ethics review.</li>
<li>Potential stability over time: As a professional networking platform, LinkedIn may be less prone to rapid shifts in usage patterns than entertainment-oriented social networks, potentially providing a relatively stable environment for recurring or longitudinal recruitment; however, this requires continued empirical monitoring.</li>
<li>Need for further evidence: Future studies should systematically evaluate recruitment efficiency, sample quality, and bias across different designs and topics and explore, where ethically permissible, whether and how organizational or contextual information can be incorporated without compromising participant protection.</li>
</ol>
</div>
<div id="further-reading" class="section level3">
<h3>Further reading</h3>
<p>Researchers interested in using social media platforms for survey recruitment may benefit from the following literature, which covers methodological, ethical, and practical aspects of recruiting survey respondents via social media advertisements. Together, these publications provide guidance on campaign design and performance, ethical and legal considerations, data quality, and the assessment of non-probability samples.</p>
<ul>
<li>Höhne, J. K., Claassen, J., Kühne, S., &amp; Zindel, Z. (2025). Social media ads for survey recruitment: Performance, costs, user engagement. <em>International Journal of Market Research</em>. <a href="https://doi.org/10.1177/14707853251367805" class="uri">https://doi.org/10.1177/14707853251367805</a></li>
<li>Pötzschke, S., Weiß, B., Daikeler, J., Silber, H., &amp; Beuthner, C. (2023). <em>A guideline on how to recruit respondents for online surveys using Facebook and Instagram: Using hard-to-reach health workers as an example</em>. Mannheim: GESIS – Leibniz Institute for the Social Sciences (GESIS Survey Guidelines). <a href="https://doi.org/10.15465/gesis-sg_en_045" class="uri">https://doi.org/10.15465/gesis-sg_en_045</a></li>
<li>Rohr, B., Felderer, B., Silber, H., Daikeler, J., Roßmann, J., &amp; Schröder, J. (2024). <em>When are non-probability surveys fit for my purpose?</em> Mannheim: GESIS – Leibniz Institute for the Social Sciences (GESIS Survey Guidelines). <a href="https://doi.org/10.15465/gesis-sg_en_050" class="uri">https://doi.org/10.15465/gesis-sg_en_050</a></li>
<li>Zimmermann, B. M., Willem, T., Bredthauer, C. J., &amp; Buyx, A. (2022). Ethical issues in social media recruitment for clinical studies: Ethical analysis and framework. <em>Journal of Medical Internet Research, 24</em>(5), e31231. <a href="https://doi.org/10.2196/31231" class="uri">https://doi.org/10.2196/31231</a></li>
<li>Zindel, Z., Kühne, S., Perrotta, D., &amp; Zagheni, E. (2025). Ad images in social media survey recruitment: What they see is what we get. <em>International Journal of Social Research Methodology</em>, 1–20. <a href="https://doi.org/10.1080/13645579.2025.2597303" class="uri">https://doi.org/10.1080/13645579.2025.2597303</a></li>
<li>Zindel, Z. (2026). Should we worry about problematic response behaviour in social media surveys? Understanding the impact of social group cues in recruitment. <em>Survey Methods: Insights from the Field</em>. <a href="https://doi.org/10.13094/SMIF-2026-00015" class="uri">https://doi.org/10.13094/SMIF-2026-00015</a></li>
<li>Zindel, Z. (2023). Social media recruitment in online survey research: A systematic literature review. <em>Methods, Data, Analyses, 17</em>(2), 207–248. <a href="https://doi.org/10.12758/mda.2022.15" class="uri">https://doi.org/10.12758/mda.2022.15</a></li>
</ul>
<!-- Add something about the instructor -->
</div>
<div id="about-the-authors" class="section level3">
<h3>About the authors</h3>
<p>Dr. Zaza Zindel <a href="mailto:zindel@dezim-institut.de"><svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg></a> <a href="https://linkedin.com/in/zaza-zindel"><svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z"/></svg></a> is a research associate at the German Centre for Integration and Migration Research (DeZIM). She is a social scientist working on survey methodology, data science, and social media research to study vulnerable, rare, and underrepresented populations and their experiences of discrimination. Her work combines methodological innovation with substantive applications using survey data, experiments, and new data sources, drawing on computational and data science approaches.</p>
<p>Dr. Lisa de Vries <a href="mailto:lisa.devries@fernuni-hagen.de"><svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg></a> <a href="https://linkedin.com/in/lisa-de-vries-38465423a"><svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3zM135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5zm282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9V416z"/></svg></a> is acting professor at FernUniversität in Hagen. Her research focuses on discrimination and inequality based on sexual orientation and gender from a quantitative perspective. Her research interests are sexual and gender minorities, labor market and educational inequalities, global developments of social acceptance and equality, and the inclusion of minority populations in research and surveys.</p>
</div>
]]>
      </description>
    </item>
    
    <item>
      <title>Using TikTok ads for survey recruitment: a step-by-step approach</title>
      <link>https://socialsciencedatalab.mzes.uni-mannheim.de/article/tiktok/</link>
      <pubDate>Fri, 15 Sep 2023 01:00:00 +0100</pubDate>
      
      <guid>https://socialsciencedatalab.mzes.uni-mannheim.de/article/tiktok/</guid>
      <description><![CDATA[
        </p>
<p>TikTok’s rapid growth and diverse user base present social science researchers with a unique opportunity to study a large and varied population, gaining valuable insights into their attitudes and behaviors. Unlike platforms such as Facebook and Instagram, TikTok’s potential in survey recruitment has been relatively underexplored. The platform’s cost-effective reach and detailed targeting parameters make it particularly appealing for reaching traditionally hard-to-reach or rare populations. Furthermore, with its video-centric format and predominantly young user base, TikTok provides a means to engage and attract respondents from younger generations to participate in online surveys. In this <a href="https://socialsciencedatalab.mzes.uni-mannheim.de/categories/tutorials/">Methods Bites Tutorial</a>, <a href="https://twitter.com/ZazaZindel">Zaza Zindel</a> and <a href="https://twitter.com/Simon_LuetkeW">Simon Lütkewitte</a> (Bielefeld University) provide a step-by-step guide on how to use TikTok ads for survey recruitment.</p>
<!-- Optional: One paragraph on learning objectives -->
<p>After reading this blog post, readers should be able to:</p>
<ul>
<li>understand how to use TikTok ads for survey participant recruitment</li>
<li>set up targeted TikTok ad campaigns</li>
<li>analyze and evaluate the performance of ad campaigns and make necessary adjustments</li>
</ul>
<!-- If applicable: Note with references on related materials -->
<p><em>Note:</em> This blog post supplements Zaza Zindel’s workshop in the <a href="https://socialsciencedatalab.mzes.uni-mannheim.de/page/events/">MZES Social Science Data Lab</a>. The original workshop materials, including slides and scripts, are available from our <a href="https://github.com/SocialScienceDataLab/social-media-ads-survey">GitHub</a>. A live recording of the workshop is available on our <a href="https://www.youtube.com/watch?v=9_z5QgMVoMU">YouTube Channel</a>.</p>
<!-- 
Generate an overview of the article 
Note: Section anchors are generated automatically from section headings as 
      hyphenated lower-case labels without special characters; e.g. a section
      title "Why R?" will result in the anchor "#why-r".
-->
<div id="overview" class="section level3">
<h3>Overview</h3>
<ol style="list-style-type: decimal">
<li><a href="#first-section"><strong>Introduction to social media survey recruitment</strong></a></li>
<li><a href="#second-section"><strong>Recruiting via TikTok – a step-by-step guide</strong></a>
<ol style="list-style-type: decimal">
<li><a href="#first-subsection">The TikTok Ads Manager</a></li>
<li><a href="#second-subsection">Setting up an ad campaign</a>
<ol style="list-style-type: decimal">
<li><a href="#first-subsubsection">Campaign level</a></li>
<li><a href="#second-subsubsection">Ad group level</a></li>
<li><a href="#third-subsubsection">Ad level</a></li>
</ol></li>
</ol></li>
<li><a href="#third-section"><strong>Application example</strong></a>
<ol style="list-style-type: decimal">
<li><a href="#third-first-subsection">Ad campaign details</a></li>
<li><a href="#third-second-subsection">Campaign results</a></li>
<li><a href="#third-third-subsection">Sample composition</a></li>
</ol></li>
<li><a href="#fourth-section"><strong>Ethical considerations</strong></a></li>
<li><a href="#fifth-section"><strong>Summary and key takeaways</strong></a></li>
<li><a href="#further-reading"><strong>Further reading</strong></a></li>
</ol>
</div>
<div id="first-section" class="section level3">
<h3>Introduction to social media survey recruitment</h3>
<p>Recruiting research participants is a crucial component of any survey study. However, traditional recruitment methods, such as random walk procedures or random digit dialing, can be time-consuming and expensive. With the growing popularity of social media, platforms like TikTok provide a unique opportunity to quickly and effectively recruit a diverse range of survey participants. TikTok is a popular social media app that allows users to create and share short-form videos with a global audience. With a self-reported number of over 1 billion active users, the platform has gained immense popularity, particularly among younger generations (<a href="https://www.tiktok.com/business/en">TikTok, n.d.</a>). TikTok is relevant and interesting for social science research due to its unique format and engaging nature. The app provides a window into the cultural and social trends of different communities, making it an attractive platform for studying human behavior and attitudes. Additionally, TikTok’s algorithm allows for a personalized feed, meaning that users are served content based on their interests and behaviors. This feature makes it an interesting platform for reaching and studying specific populations.</p>
<p>As a potential tool for survey recruitment, TikTok ads offer several advantages. The platform’s vast and diverse user base provides researchers with access to a broad range of potential participants, while its targeting options allow them to reach very specific populations that may be difficult to reach through more traditional recruitment channels. Unlike social media platforms like Facebook and Instagram, TikTok’s structure revolves around short-form videos set to music, offering a unique and visually engaging format that researchers can utilize to capture users’ attention and encourage their participation. By leveraging visually appealing videos, researchers can create compelling survey invitations, enhancing the overall survey experience. Furthermore, TikTok’s predominantly young user base sets it apart from other platforms, making it particularly valuable for studies targeting younger demographics. This demographic distinction allows researchers to gain insights from a generation with unique attitudes, behaviors, and perspectives, ultimately contributing to a more comprehensive understanding of societal trends and dynamics.</p>
</div>
<div id="second-section" class="section level3">
<h3>Recruiting via TikTok – a step-by-step guide</h3>
<p>In the following section, we will walk you through the step-by-step process of setting up advertisements on TikTok. We will highlight the important factors you need to consider in order to generate a diverse and comprehensive sample for a web survey.</p>
<div id="first-subsection" class="section level5">
<h5>The TikTok Ads Manager</h5>
<p>Having an active TikTok account is the first step towards advertising and recruiting survey participants on TikTok. This is because only individuals and organizations with registered business accounts are allowed to run ads on the platform. To register for a business account, go to the <a href="https://www.tiktok.com/business/en">TikTok for Business website</a> and proceed by clicking the “Create Now” button (see Figure <a href="#fig1">1</a>).</p>
<a name="fig1"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig1"></span>
<img src="/../../../../../article/tiktok_files/images/TikTokforBusiness.png" alt="Starting page of TikTok for Business website." width="80%" />
<p class="caption">
Figure 1: Starting page of TikTok for Business website.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.tiktok.com/business/en">Tik Tok for Business</a>
</sup></sub>
</p>
</div>
<p>From there, follow the prompts to sign up for a new account. You will need to provide basic information such as your email address and a password to create an account. Once you have an account, you can proceed to set up your business profile (see Figure <a href="#fig2">2</a>). This includes providing information such as your company name, industry and country. Make sure to accurately and clearly represent your business, as this will help to build trust with potential survey participants.</p>
<a name="fig2"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig2"></span>
<img src="/../../../../../article/tiktok_files/images/BusinessAccount2.PNG" alt="TikTok for Business profile." width="40%" />
<p class="caption">
Figure 2: TikTok for Business profile.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://www.tiktok.com/business/en">Tik Tok for Business (after login)</a>
</sup></sub>
</p>
</div>
</div>
<div id="second-subsection" class="section level5">
<h5>Setting up an ad campaign</h5>
<p>When creating an advertising campaign on TikTok, it is important to familiarize yourself with the campaign structure. The campaign settings are structured into three distinctive hierarchical levels: campaign level, ad group level, and ad level (see Figure <a href="#fig3">3</a>). Within the ad campaign, you create ad groups, which serve as subdivisions allowing for further targeting refinement. Each ad group can contain multiple individual ads that serve as the creative content displayed to TikTok users. This means that within a single ad campaign, you can create multiple ad groups, and within each ad group, there can be multiple ads. Each level requires making informed choices and providing necessary information to ensure the effectiveness of your ad campaign.</p>
<a name="fig3"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig3"></span>
<img src="/../../../../../article/tiktok_files/images/hierarchy.png" alt="TikTok ad hierarchy." width="80%" />
<p class="caption">
Figure 3: TikTok ad hierarchy.
</p>
</div>
<div id="first-subsubsection" class="section level6">
<h6>Campaign level</h6>
<p>At the campaign level, the first step is to choose the campaign objective that best aligns with your advertising goal. This decision will determine how TikTok will optimize the ad delivery. TikTok offers three main categories of campaign objectives: awareness, consideration, and conversion (see Figure <a href="#fig4">4</a>). Each category serves a distinct purpose in terms of user interaction with your ad and the desired actions you want users to take.</p>
<a name="fig4"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig4"></span>
<img src="/../../../../../article/tiktok_files/images/setup3_1.png" alt="Selecting an advertising objective." width="80%" />
<p class="caption">
Figure 4: Selecting an advertising objective.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
<p>Firstly, awareness campaigns are aimed at sparking interest and enhancing the visibility of a brand, product, or service. These campaigns are designed to reach as many people as possible and create a lasting impression of a brand. Secondly, consideration campaigns are focused on driving engagement and interest. This includes encouraging users to visit a website, interact with content, or follow an account. These campaigns are designed to provide users with more information about a brand, product, or service, and to motivate them to take a specific desired action. Lastly, conversion campaigns are designed to drive users towards a specific action, such as making a purchase or downloading an app.</p>
<p>When creating a TikTok ad campaign to recruit survey participants, we highly recommend setting the campaign objective to “Traffic”, which is categorized under consideration. This objective focuses on driving traffic to a designated landing page, such as your survey website. By selecting this objective, your ad campaign will be optimized to effectively direct users to your survey website, increasing the likelihood of attracting potential participants to take part in your survey. TikTok’s machine learning algorithms will be leveraged to identify users within the target audience who are most likely to take the desired action of clicking on the ad and visiting your landing page.</p>
<p>After selecting the campaign objective, you can proceed to define some essential settings of your campaign. These settings include providing a campaign name and defining a maximum budget for the entire campaign (see Figure <a href="#fig5">5</a>). While the maximum budget is not mandatory, we recommend setting it as a precautionary measure to prevent your campaign from exceeding the spending limit of your research project.</p>
<a name="fig5"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig5"></span>
<img src="/../../../../../article/tiktok_files/images/Setup3.2.png" alt="Defining campaign name and campaign budget." width="70%" />
<p class="caption">
Figure 5: Defining campaign name and campaign budget.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
</div>
</div>
<div id="second-subsubsection" class="section level5">
<h5>Ad group level</h5>
<p>At this level, you can create multiple ad groups within a campaign, each with its own settings tailored to the target audience, thereby enhancing ad effectiveness. You can specify various settings unique to each ad group. These include the optimization location, placements, targeting, budget and schedule, bidding, and optimization. First, set the optimization location, i.e. where to direct traffic. For survey recruitment, the optimization location should be set to the survey website where participants can complete the survey. Under the traffic objective, ad placement in Germany is limited to TikTok and Pangle (see Figure <a href="#fig6">6</a>). While TikTok is the primary platform where ads will appear, Pangle is an ad network that provides access to third-party apps. We recommend to choose TikTok instead of Pangle, as TikTok has a larger user base and provides more advanced targeting options, which can help you reach a more specific audience. For other countries, other ad placements are available (for an overview see <a href="https://ads.tiktok.com/help/article/placements-available-locations">TikTok (n.d.)</a>). Additionally, within the placements section, you have the option to customize your ad settings to control the level of user engagement. Consider deactivating comments to avoid harmful or violent reactions, disabling video downloads to protect your content, and restricting video sharing to ensure that your ads are exclusively viewed within the TikTok platform.</p>
<a name="fig6"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig6"></span>
<img src="/../../../../../../article/tiktok_files/images/Setup5.PNG" alt="Defining placement options." width="70%" />
<p class="caption">
Figure 6: Defining placement options.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
<p>Targeting is another crucial step at the ad group level where you define the audience that your ad will be shown to. TikTok provides a range of targeting options, including demographic information, interests, behaviors, and custom audiences. You should define the target audience based on your research objectives and select the appropriate targeting options to effectively reach your desired audience. Take advantage of the wide range of targeting options available when running TikTok ads. Here are some examples:</p>
<ul>
<li>Demographic targeting: Gender (binary), age groups, and language preferences (see Figure <a href="#fig7">7</a>).</li>
<li>Geographic targeting: Countries/regions you can target using “Location” depending on the country/region associated with your TikTok For Business account. For a comprehensive list of all available locations per country, see <a href="https://ads.tiktok.com/help/article/placements-available-locations">TikTok (n.d.)</a>.</li>
<li>Interest-based targeting: Reach users based on their inferred interests, derived from their activity on TikTok (see Figure <a href="#fig8">8</a>).</li>
<li>Behavioral targeting: Reach users based on their behavior on TikTok, such as the videos they have liked, shared, or commented on (see Figure <a href="#fig8">8</a>).</li>
</ul>
<a name="fig7"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig7"></span>
<img src="/../../../../../article/tiktok_files/images/Setup6_1.PNG" alt="Defining demographic targeting parameters." width="70%" />
<p class="caption">
Figure 7: Defining demographic targeting parameters.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
<a name="fig8"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig8"></span>
<img src="/../../../../../article/tiktok_files/images/Setup7_1.PNG" alt="Defining interest- and behavior-based targeting." width="70%" />
<p class="caption">
Figure 8: Defining interest- and behavior-based targeting.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
<p>It is crucial to ensure compliance with TikTok’s advertising policies and avoid any form of discriminatory targeting towards protected groups. TikTok maintains strict advertising policies to uphold community standards and legal requirements. Advertisers are expected to follow these policies, which are regularly updated, in order to maintain the quality and integrity of the TikTok platform. For a detailed description of all prohibited and restricted content, please refer to <a href="https://ads.tiktok.com/help/article/tiktok-advertising-policies-ad-creatives-landing-page">TikTok (n.d.)</a>.</p>
<p>The next step at the ad group level is to set a budget and schedule. You should define a daily budget for each ad group and specify the display schedule for each group. TikTok offers an estimation of an ad group’s potential reach based on the selected targeting options. This estimation can assist in determining an appropriate budget for your campaign (see Figure <a href="#fig9">9</a>).</p>
<a name="fig9"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig9"></span>
<img src="/../../../../../article/tiktok_files/images/Setup11.PNG" alt="Estimated audience reach." width="25%" />
<p class="caption">
Figure 9: Estimated audience reach.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
<p>Once you have set the budget and schedule for your ad group, the next step is to define the options in Bidding &amp; Optimization (see Figure <a href="#fig10">10</a>). This step determines how the ad will be delivered and optimized for maximum effectiveness. On TikTok, you have the choice between two optimization goals: Click and landing page view. The default and recommended optimization goal is Click, which focuses on driving users to click on the ad and visit the website. Alternatively, the landing page view optimization goal targets users who not only click on the ad but also waits for the landing page to load completely. This is particularly useful for campaigns that require users engagement with the content on the landing page.</p>
<a name="fig10"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig10"></span>
<img src="/../../../../../article/tiktok_files/images/Setup10_1.PNG" alt="Defining bidding and optimization options." width="70%" />
<p class="caption">
Figure 10: Defining bidding and optimization options.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: <a href="https://ads.tiktok.com/">Tik Tok Ads Manager (after login)</a>
</sup></sub>
</p>
</div>
<p>Furthermore, you also have the option to select between two bid strategies: lowest cost and bid cap. The lowest cost bidding strategy is a spend-based approach, where TikTok’s algorithm automatically adjusts the bid to generate as many results as possible at the lowest cost per result. This strategy is ideal for most campaigns. However, if you want to have more control over the budget allocation, you can choose the bid cap option. With bid cap, you can set a maximum bid amount for each link click, ensuring that the cost per result remains below your predetermined limit. This strategy can be particularly beneficial for campaigns with limited budgets or when you want to ensure that you are not overpaying for each link click.</p>
</div>
<div id="third-subsubsection" class="section level5">
<h5>Ad level</h5>
<p>On the ad level, you can create a captivating advertisement that will be displayed to the target audience. The goal is to make the ad highly appealing and drive engagement among your target population. To achieve this, your ad should be eye-catching, and effectively convey the purpose of the survey. Given that TikTok is a platform centered around videos, we highly recommend incorporating a short and captivating video into your ad. The video should grab viewers’ attention and motivate them to take action. While the ad format can also be a single image or image carousel, it is advisable to include at least some dynamic elements to align with the platform’s style. Moreover, you must keep in mind that TikTok ads are primarily optimized for mobile devices. Hence, the ad should be visually appealing and easily viewable on smartphones, ensuring a seamless user experience.</p>
<a name="fig11"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig11"></span>
<img src="/../../../../../article/tiktok_files/images/Ad_example.png" alt="Example ad design." width="80%" />
<p class="caption">
Figure 11: Example ad design.
</p>
</div>
<div style="text-align: right">
<p>
<sub><sup>
Source: Image by Macrovector via stock.adobe.com</a>
</sup></sub>
</p>
</div>
<p>In addition, you have the option to choose from various call-to-action buttons, such as “Learn More” or “Sign Up”, which are designed to encourage users to take action. When users click on the button, they will be directed to the survey (see Figure <a href="#fig11">11</a>). Furthermore, it is important to ensure that the ad text and description convey a clear and concise message that motivates users to participate in the survey.</p>
<p>Once the ad campaign is created, it can be launched. However, it is important to note that TikTok will review the ads to ensure compliance with their guidelines and policies. It is essential to avoid displaying prohibited products or services, such as live animals, adult sexual products, casinos, military gear, or political candidates or parties, in the ad creatives or on the landing pages. A comprehensive list of prohibited content can be found at <a href="https://ads.tiktok.com/help/article/tiktok-advertising-policies-industry-entry">TikTok (n.d.)</a>.</p>
<p>Once the ads are live, you can utilize TikTok’s ad manager tool to track their performance. Monitoring the number of views, clicks, and conversions is essential to assess the reach, engagement, and overall effectiveness of the ads. This valuable information can be used to make necessary adjustments to the ad settings and enhance the performance of future ads.</p>
</div>
</div>
<div id="third-section" class="section level3">
<h3>Application example</h3>
<p>The following section presents the results of an exemplary TikTok advertising campaign conducted as part of a master’s seminar at Bielefeld University. In this seminar, students were tasked with designing a questionnaire aligned with their individual research interests. The main topics were attitudes toward refugees and youth crime in the UK and Germany. To efficiently and cost-effectively generate a substantial sample size, the study was promoted through TikTok ads.</p>
<div id="third-first-subsection" class="section level4">
<h4>Ad campaign details</h4>
<p>In accordance with the previously discussed hierarchical structure, various settings were configured at the three different levels of the advertising campaign (see Table <a href="#table1">1</a>).</p>
<a name="table1"></a>
<caption>
Table 1: Setup of example campaign
</caption>
<iframe src="hierarchy/index.html" frameborder="no" width="100%" height="840px" frameborder="no">
</iframe>
<p>At the campaign level, we opted for a consideration advertising objective, specifically “Traffic”, with the aim of driving traffic to our survey website. We set the maximum budget of 400.00 EUR for the entire study. Moving on to the ad group level, we created three distinct ad groups: one targeting English-speaking individuals in the UK (ad group: UK), another targeting German-speaking individuals in Germany (ad group: DE1), and a third focusing on English speaking-individuals in Germany (ad group: DE2). While these ad groups differed in terms of location and language targeting, certain settings were kept consistent across all three: age range, gender, maximum budget per ad group, schedule, delivery, bid strategy, optimization goal, and billing event. For each ad group, we allocated a maximum budget of 130.00 EUR, aiming to evenly distribute our budget across the ad groups. The recruitment phase was scheduled to run from August 6th, 2022, at 16:00 to August 9th, 2022, at 16:00. We configured all ad groups to deliver ads throughout the day, and the optimization goal for each group was to maximize clicks on the advertisements. To minimize costs while maximizing the number of clicks, we selected the “lowest cost” bid strategy. Based on our settings, the billing event was set to cost per click (CPC).</p>
<p>We created a total of 15 ads, with five ads designed under each of the three ad groups. These ads included either a video or an image with animations. As an example, Video <a href="#video1">1</a> illustrates an ad tailored to address the topic of juvenile delinquency, and was specifically targeted to the English-speaking target population. In addition to the visuals, each ad featured our custom identity (<span class="citation">@SozPolStudyUniBielefeld</span>), license-free background music provided through TikTok, and concise yet meaningful ad texts, such as “Take our short survey!” in the corresponding language of the target group. To encourage user engagement, we utilized the “Learn more” call-to-action button. When users clicked on this button, they were redirected to our survey page, and we employed URL parameters to track from which respective ad participation in the survey took place.</p>
<p><a name="video1"></a></p>
<center>
<video width="500" height="500" controls>
  <source src="/../../../../../article/tiktok_files/Ads/YD1UK-YD1DE2.mp4" type="video/mp4">
</video>
</center>
<br>
<caption>Video 1: Example of an ad video.</caption>
</br>
<div style="text-align: right">
<p>
<sub><sup>
Source: Video by cottonbro via Canva.com</a>
</sup></sub>
</p>
</div>
</div>
<div id="third-second-subsection" class="section level4">
<h4>Campaign results</h4>
<p>Table <a href="#table2">2</a> provides an overview of the ad campaign’s performance metrics. Over the course of the 4-day campaign, it reached a total of 117,626 TikTok users. The videos or animated images were played 189,100 times, with the ads being displayed 205,071 times, as reported by the TikTok Ad Manager. The campaign incurred a total cost of 387.48 EUR and generated 3,603 clicks on the call-to-action button, representing a click-through rate of 1.76 percent. The average cost per click was 0.11 EUR, which is comparable to recruitment via other social media platforms like Facebook or Instagram (cf. e.g., <a href="https://doi.org/10.12758/mda.2022.15">Zindel 2022</a>).</p>
<p>Analyzing the ad performance reveals variation depending on the target groups. Ads placed in Germany (ad groups: D1 &amp; D2) reached a higher overall number of impressions compared to the UK. This discrepancy can be attributed to the less developed TikTok ads market in Germany compared to the UK, resulting in lower costs for link clicks due to reduced ad competition. Consequently, the same ad budget resulted in varying levels of impressions. Figure <a href="#fig12">12</a> illustrates the cost per click (CPC) by ad group. Initially, during the learning phase of the advertising algorithm, ads in the UK were slightly more expensive than those in Germany. Moreover, the ad group targeting English speakers in Germany (ad group: DE2) exhibited the highest cost per click over the short duration of the campaign. This can be attributed to the relatively lower proportion of TikTok users in Germany who use the platform in English compared to those who use it in German. Targeting a smaller user base incurs higher costs to reach a significant number of users, as reflected in the number of impressions in Table <a href="#table2">2</a>.</p>
<a name="fig12"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig12"></span>
<img src="/../../../../../article/tiktok_files/images/CPC.png" alt="Average cost per click (CPC) for each ad group." width="70%" />
<p class="caption">
Figure 12: Average cost per click (CPC) for each ad group.
</p>
</div>
<p>Examining the metrics distribution in Table <a href="#table2">2</a> highlights significant variations in ad performance. Some ads were displayed and viewed more frequently than others within the same ad group. For instance, in ad group DE2, the ad YD2DE2, one of two ads that focused on the topic of youth delinquency (“YD” for short), had the highest allocation rate determined by the advertising algorithm, resulting in the highest number of clicks on the call-to-action button within this group. When considering the click-through-rate (CTR) for all ads, which represents the conversion rate from impressions to clicks, ads served more frequently tended to have comparatively higher CTRs. However, it is important to note that these ads may not necessarily have the highest CTR within their respective ad groups. For example, in the UK ad group, the ad RF2UK had a CTR of 2.66 percent, surpassing the most frequently displayed ad YD2UK with a CTR of 2.22 percent. This discrepancy arises from the fact that ads that initially garner significant attention in the initial learning phase of the advertising algorithm may not necessarily sustain the highest level of interest from the target audience throughout the entire ad runtime <a href="https://ads.tiktok.com/help/article/learning-phase?lang=en">TikTok (n.d.)</a>.</p>
<a name="table2"></a>
<caption>
Table 2: Results of ad performance.
</caption>
<iframe src="table2_new/index.html" frameborder="no" width="100%" height="400px" frameborder="no">
</iframe>
<p>Table <a href="#table3">3</a> presents the conversion of link clicks per ad into started and completed interviews, along with the corresponding total cost and cost per completed interview. Out of the 3,603 link clicks, 1,846 (51.29 percent) resulted in started interviews, and a total of 500 interviews (13.88 percent) were completed. With a total campaign cost of 387.48 EUR, the average cost per completed interview was 0.77 EUR. Notably, variations can be observed across the ad groups. The ad group targeting German-speaking individuals in Germany (DE1) received the highest number of link clicks (n=1,676), resulting in the highest number of started interviews (n=983; 58.65 percent), as well as completed interviews (n=265; 15.81 percent). This specific target group was also the most cost-effective to reach, with an average cost of 0.49 EUR per completed interview. In contrast, the average cost per completed interview was 0.99 EUR for the DE2 ad group and 1.23 EUR for the UK group. This disparity highlights the differences in costs discussed earlier, which are influenced by the bidding system and the ad algorithm.</p>
<a name="table3"></a>
<caption>
Table 3: Results of survey performance.
</caption>
<iframe src="table3/index.html" frameborder="no" width="100%" height="400px" frameborder="no">
</iframe>
</div>
<div id="third-third-subsection" class="section level4">
<h4>Sample composition</h4>
<p>The final sample comprises three subsamples derived from the three different ad groups: English-speaking individuals in the UK (ad group: UK), German-speaking individuals in Germany (ad group: DE1), and English-speaking individuals in Germany (ad group: DE2). Group DE1 accounts for 265 completed interviews, representing 53.00 percent of all interviews. Group DE2 includes 130 interviews (26.00 percent) and group UK has 105 interviews (21.00 percent). The average age of the respondents is 37.22 years (see Figure <a href="#fig13">13</a>), which is higher than expected considering TikTok’s popularity among younger generations. However, it is important to note that the average age in our sample still falls below the overall average age of 44.7 in the German population (<a href="https://de.statista.com/statistik/daten/studie/1084430/umfrage/durchschnittsalter-der-bevoelkerung-in-deutschland/#:~:text=Zum%20Ende%20des%20Jahres%202021,der%20m%C3%A4nnlichen%20Bev%C3%B6lkerung%20in%20Deutschland">Statistische Ämter des Bundes und der Länder, 2022</a>) and the median age for the UK (<a href="https://de.statista.com/statistik/daten/studie/200671/umfrage/durchschnittsalter-der-bevoelkerung-in-grossbritannien/">UN DESA, 2022</a>).</p>
<a name="fig13"></a>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:fig13"></span>
<img src="../../../../../../../../article/tiktok_files/images/Age.png" alt="Age distribution in overall sample." width="50%" />
<p class="caption">
Figure 13: Age distribution in overall sample.
</p>
</div>
<p>The average age within the DE1 ad group is 47.74 years, whereas it is 23.99 years for the DE2 ad group and 25.53 years for individuals in the UK. This indicates a strong selection bias towards older individuals in the German-speaking group, while the other ad groups exhibit a strong bias towards younger participants. Table <a href="#table4">4</a> compared the age distribution of each ad group with the ad allocation information provided by TikTok. These selection effects are primarily influenced by the ad algorithm. Therefore, for studies aiming to achieve a demographically balanced age distribution, it is recommended to further stratify ad groups and run specific ads targeted at different age groups to mitigate similar age biases.</p>
<a name="table4"></a>
<caption>
Table 4: Age group distribution by ad groups.
</caption>
<iframe src="table4/index.html" frameborder="no" width="100%" height="480px" frameborder="no">
</iframe>
<p>Regarding gender distribution, a larger proportion (61.52 percent) of male participants was identified in the sample, while only 27.05 percent identified as female (see Table <a href="#table5">5</a>). Additionally, 60 individuals (11.43 percent) in the sample reported identifying with a different gender. A comparison of the sample data with the information from the TikTok Ads-Manager reveals that this effect can also be partially attributed to the allocation of the ads by the advertising algorithm. The DE1 ad group was predominantly delivered to men, while the DE2 and UK ad groups did not exhibit such a strong gender effect in their ad display. It appears that a large proportion of respondents, assumed to be female, chose the third response option, “another gender”. However, it is important to note that we lack individual gender information within the TikTok platform.</p>
<a name="table5"></a>
<caption>
Table 5: Gender distribution by ad groups.
</caption>
<iframe src="table5/index.html" frameborder="no" width="100%" height="320px" frameborder="no">
</iframe>
</div>
</div>
<div id="fourth-section" class="section level3">
<h3>Ethical considerations</h3>
<p>When considering the use of TikTok for survey recruitment and advertisements in general, several ethical implications merit careful consideration. TikTok, a Chinese company owned by ByteDance with established ties to the Chinese state, raises pertinent concerns. Purchasing advertising space on this platform contributes to TikTok’s financial revenue, strengthening its market position as one of the fastest-growing social media platforms. Due to its indirect connection to the Chinese state, numerous government agencies, such as the European Commission and the European Council, have banned TikTok from their official devices. Moreover, some countries have opted for a complete prohibition of TikTok due to concerns encompassing cyber security, privacy, and misinformation.</p>
<p>With the approach outlined in this blog post, TikTok does not have access to the survey data at any time. The social media platform remains incapable of identifying which users have participated in our survey. To ensure that the survey data remains inaccessible to third parties at all times, we strongly advise against the utilization of tracking tools provided by TikTok, as well as other platforms.</p>
<p>Nonetheless, it is worth noting that TikTok creates interest profiles of users based on their interaction with advertised content – a practice also common among other social media platforms. However, considering TikTok’s indirect link to the Chinese state, the possibility that these interest profiles could be accessed by Chinese authorities cannot be disregarded. This potential vulnerability is especially concerning for marginalized population groups, such as LGBTQI* individuals. While TikTok may lack specific information regarding the gender or sexual identity of its users, it can infer from users’ interactions with content related to LGBTQI* topics that these users at least hold an interest in such matters.</p>
<p>For all researchers intending to establish financial collaborations with TikTok –- whether to recruit survey participants or promote other content –- we strongly recommend a preliminary consultation with their respective institutions, employers, and ethics committees. This step will help determine the feasibility of such projects within the ethical framework.</p>
</div>
<div id="fifth-section" class="section level3">
<h3>Summary and key takeaways</h3>
<p>Recruiting survey participants through TikTok ads offers several advantages for researchers. TikTok’s vast and diverse user base allows access to a broad range of potential participants, and its targeting options enable reaching specific populations that may be challenging to reach through traditional methods. By following our presented step-by-step guide for setting up advertisements on TikTok, you can effectively generate a diverse and inexpensive sample for your web surveys. The example of a TikTok advertising campaign demonstrates the platform’s potential in efficiently and cost-effectively recruiting survey participants. We delved into its unique features, demographics, and engagement levels, considering its suitability for reaching a diverse audience. Additionally, we examined the advantages and disadvantages of utilizing TikTok, highlighting its ability to generate rapid and widespread exposure while also acknowledging the challenges it presents, such as limited insights in the algorithmic ad allocation and potential for high selectivity.</p>
<p>Overall, the key takeaways from this tutorial are as follows:</p>
<ul>
<li>TikTok offers a vast user base with diverse demographics, making it a promising platform for survey recruitment efforts aimed at reaching a wide audience.</li>
<li>Ad campaigns on TikTok are structured into campaign levels, ad group levels, and ad levels, each requiring informed choices and necessary information for effectiveness.</li>
<li>TikTok provides various targeting options, including demographics, interests, behaviors, and custom audiences, to reach the desired audience.</li>
<li>Ad creatives on TikTok should be eye-catching, incorporate videos or dynamic elements, and have clear ad text and call-to-action buttons.</li>
<li>It is essential to carefully monitor and address potential ethical concerns, such as data privacy and user consent, when utilizing TikTok for survey recruitment.</li>
<li>Monitoring the performance of TikTok ads through the ad manager tool is crucial for assessing reach, engagement, and effectiveness, especially when it becomes apparent that certain ad groups outperform others.</li>
<li>The distribution of ads by the algorithm may lead to overrepresentation of different subgroups. For demographically more comparable population distributions, it is recommended to stratify the ads using the targeting parameters.</li>
</ul>
</div>
<div id="further-reading" class="section level3">
<h3>Further reading</h3>
<p>Here are several publications that offer a valuable introduction to the topic of social media ads for survey respondent recruitment.These publications provide valuable insights into the use of social media ads for recruiting survey respondents and serve as a great starting point for exploring this subject further:</p>
<ul>
<li>Iannelli, L., Giglietto, F., Rossi, L., &amp; Zurovac, E. (2020). Facebook Digital Traces for Survey Research: Assessing the Efficiency and Effectiveness of a Facebook Ad–Based Procedure for Recruiting Online Survey Respondents in Niche and Difficult-to-Reach Populations. Social Science Computer Review, 38(4), 462–476. <a href="doi:https://doi.org/10.1177/0894439318816638" class="uri">doi:https://doi.org/10.1177/0894439318816638</a></li>
<li>Kühne, S., &amp; Zindel, Z. (2020). Using Facebook and Instagram to Recruit Web Survey Participants: A Step-by-Step Guide and Application. Survey Methods: Insights from the Field. <a href="doi:https://doi.org/10.13094/SMIF-2020-00017" class="uri">doi:https://doi.org/10.13094/SMIF-2020-00017</a></li>
<li>Neundorf, A., &amp; Öztürk, A. (2021, July 26). How to Improve Representativeness and Cost-effectiveness in Samples Recruited through Meta: A Comparison of Advertisement Tools. url:<a href="https://doi.org/10.31219/osf.io/3g74n" class="uri">https://doi.org/10.31219/osf.io/3g74n</a></li>
<li>Neundorf, A., &amp; Öztürk, A. (2021, December 9). Recruiting Research Participants through Facebook Advertisements: A Handbook. url:<a href="https://doi.org/10.31219/osf.io/87rg3" class="uri">https://doi.org/10.31219/osf.io/87rg3</a></li>
<li>Zindel, Z. (2022). Social Media Recruitment in Online Survey Research: A Systematic Literature Review. methods, data, analyses, 0, 42. <a href="doi:https://doi.org/10.12758/mda.2022.15" class="uri">doi:https://doi.org/10.12758/mda.2022.15</a></li>
</ul>
<p>We would also like to highlight the excellent webinar series of the <a href="https://www.gla.ac.uk/research/az/democracyresearch/dataandmethods/socialmediaasaresearchtool/webinarseries/">DEMED project</a>. This series offers a range of insightful presentations that delve into the advantages of social media for survey research.</p>
<!-- Add something about the instructor -->
</div>
<div id="about-the-authors" class="section level3">
<h3>About the authors</h3>
<p>Zaza Zindel <a href="mailto:zaza.zindel@uni-bielefeld.de"><svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg></a> <a href="https://twitter.com/ZazaZindel"><svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg></a> is a research assistant and a doctoral researcher in sociology at Bielefeld University (Germany) with a specialization in survey research. Her dissertation centers around the utilization of social media as a means to recruit rare populations for web surveys. Her research interests encompass survey methodology, the potential of social media for empirical social research, and the exploration of new technologies to improve statistical representation of marginalized, vulnerable, or rare population groups.</p>
<p>Simon Lütkewitte <a href="mailto:simon.luetkewitte@uni-bielefeld.de"><svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M64 112c-8.8 0-16 7.2-16 16v22.1L220.5 291.7c20.7 17 50.4 17 71.1 0L464 150.1V128c0-8.8-7.2-16-16-16H64zM48 212.2V384c0 8.8 7.2 16 16 16H448c8.8 0 16-7.2 16-16V212.2L322 328.8c-38.4 31.5-93.7 31.5-132 0L48 212.2zM0 128C0 92.7 28.7 64 64 64H448c35.3 0 64 28.7 64 64V384c0 35.3-28.7 64-64 64H64c-35.3 0-64-28.7-64-64V128z"/></svg></a> <a href="https://twitter.com/Simon_LuetkeW"><svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg></a> is a research assistant at Bielefeld University (Germany) specializing in empirical social science research, particularly quantitative methods . He is also a PhD student at the Bielefeld Graduate School for History and Sociology (BGHS). His research focuses on values and attitudes within the sphere of sports. Methodologically, he is interested in applying statistical methods such as multilevel modeling, matching techniques, and social network analysis. In his dissertation project, he utilizes quantitative methods to examine the relationship between sports participation and gender-related attitudes and beliefs.</p>
</div>
]]>
      </description>
    </item>
    
    <item>
      <title>Survey data collection from start to finish</title>
      <link>https://socialsciencedatalab.mzes.uni-mannheim.de/article/survey-data-collection/</link>
      <pubDate>Mon, 11 Apr 2022 01:00:00 +0100</pubDate>
      
      <guid>https://socialsciencedatalab.mzes.uni-mannheim.de/article/survey-data-collection/</guid>
      <description><![CDATA[
        </p>
<!-- Article text -->
<p>Surveys have long been a staple of social science research on individuals’ attitudes and behaviors. In recent years, however, we have witnessed a strong shift from secondary analyses of large general social surveys toward smaller, more targeted primary data collections. This development has been accompanied by the increasing availability of affordable and easy-to-implement surveys using online access panels. While the entry barriers to original survey-based research are now likely lower than ever before, it still comes with notable methodological, administrative, and logistic challenges. To help aspiring survey researchers navigate this process, this <a href="https://socialsciencedatalab.mzes.uni-mannheim.de/categories/instructionals/">Methods Bites Instructional</a> by <a href="https://twitter.com/hellyer_josh">Joshua Hellyer</a> (MZES, University of Mannheim) provides a comprehensive guide to survey data collection with online access panels.</p>
<p>Reading this blog post, you will learn:</p>
<ul>
<li>The steps involved in planning and conducting a survey experiment</li>
<li>Best practices in reproducibility</li>
<li>Tips on completing your application for ethical approval</li>
<li>Pros and cons of working with an online access panel</li>
</ul>
<p><em>Note:</em> This blog post provides a summary of Joshua’s and <a href="https://twitter.com/JohannaGereke">Johanna Gereke</a>’s input talk “Survey data collection from start to finish: Designing &amp; executing reproducible research with an online access panel” in the <a href="https://socialsciencedatalab.mzes.uni-mannheim.de/page/events/">MZES Social Science Data Lab</a> in Spring 2022. The original workshop materials, including slides and scripts, are available from our <a href="https://github.com/SocialScienceDataLab/survey-data-collection">GitHub</a>.
A live recording of the talk is available on our <a href="https://youtu.be/NZhzWVYeCLI">YouTube Channel</a>.</p>
<div id="overview" class="section level3">
<h3>Overview</h3>
<ol style="list-style-type: decimal">
<li><a href="#introduction"><strong>Introduction</strong></a></li>
<li><a href="#research-design"><strong>Research Design</strong></a>
<ol style="list-style-type: decimal">
<li><a href="#sampling-and-power">Sampling and Power</a></li>
<li><a href="#ethical-approval">Ethical Approval</a></li>
</ol></li>
<li><a href="#pre-registration"><strong>Pre-Registration</strong></a></li>
<li><a href="#data-collection"><strong>Data Collection</strong></a>
<ol style="list-style-type: decimal">
<li><a href="#online-access-panels">Online Access Panels</a></li>
<li><a href="#programming">Programming</a></li>
<li><a href="#pilot-testing-and-data-collection">Pilot Testing and Data Collection</a></li>
</ol></li>
<li><a href="#data-and-code-sharing"><strong>Data and Code Sharing</strong></a></li>
<li><a href="#conclusion"><strong>Conclusion</strong></a></li>
</ol>
</div>
<div id="introduction" class="section level3">
<h3>Introduction</h3>
<p>In this blog post, I will walk you through the process of fielding your own survey experiment using an online access panel. Online access panels are pre-selected groups of internet users who are paid to participate in various surveys, often including market research as well as scientific studies. You may have heard of panel providers like <a href="https://www.bilendi.de/">Bilendi/Respondi</a>, <a href="https://www.dynata.com/">Dynata</a>, <a href="https://www.kantarpublic.com/de">Kantar</a>, and <a href="https://yougov.de/">YouGov</a> that are frequently used in social science research. They have become popular among social scientists because they are a relatively fast, easy, and cheap way to reach a sample of the general population, but there are certainly pros and cons that you should consider, as I will describe in this post.</p>
<p>Throughout this post, I will use a recent data collection as an example, a survey experiment that I conducted with <a href="https://johannagereke.com">Johanna Gereke</a> (whose expertise I rely on extensively for this piece) and several other colleagues about demographic threat and how it affects group boundaries in the German context <span class="citation">(Gereke et al. <a href="#ref-gereke_demographic_2022" role="doc-biblioref">2022</a>)</span>. In some ways, this project is unique: it was completed as part of a replication seminar at the University of Mannheim, in which we worked with 12 Master’s and PhD students who helped plan the survey, analyze data, and write the first draft of our paper. But in many other ways, our experiences should apply to a wide range of potential surveys, and I hope that you will find it useful no matter the size of your research team or the topic you plan to study.</p>
<p>So, what steps are required to plan and execute a survey experiment? Your experience may vary depending on the complexity of your planned survey as well as the population you plan to sample from, but you can get an overview of our recent experience in the timeline shown in the figure below. This project was completed relatively quickly, going from initial conception to journal submission in just over 6 months. This timeline was intentionally fast, as we wanted the students to experience the whole data collection process in a single semester, but this is also an example of just how fast you can collect data using an online access panel. As you can see, the most time-consuming part of our research was designing the survey, but also note that ethical approval can take some time (more on this later). While your workflow may look a bit different, I would generally recommend proceeding in a similar order. In particular, ethical approval must come early in the process as you need this to collect any data. Tackling elements like this (and also procurement) that are out of your control early on can save you from delays later.</p>
<div class="figure" style="text-align: center"><span style="display:block;" id="fig:graphic1"></span>
<img src="/../../../../../article/survey-data-collection_files/figure-html/graphic1-1.png" alt="Project timeline." width="100%" />
<p class="caption">
Figure 1: Project timeline.
</p>
</div>
</div>
<div id="research-design" class="section level3">
<h3>Research Design</h3>
<p>For our seminar, we chose to first replicate and then extend Maria Abascal’s recent work on group boundaries in the U.S. context <span class="citation">(Abascal <a href="#ref-abascal_contraction_2020" role="doc-biblioref">2020</a>)</span>. She finds that when white Americans are confronted with information about demographic threat, i.e. that they will no longer make up a majority of the U.S. population, they are more exclusive about who they see as “White”. We tested whether the same effect could be found in Germany, randomly assigning each of our participants to see one of two demographic projections that either predicted that people with a migration background would make up an equal share of the population in the future (treatment), or that they would remain a minority (control). After seeing this, participants rated pictures of 18 people as having a migration background or not. We hypothesized that following Abascal, the demographic threat treatment would make people more likely to classify people who appear ethnically ambiguous as having a migration background.</p>
<div id="questionnaire" class="section level5">
<h5>Questionnaire</h5>
<p>Because our experiment was based on previous research, designing our survey was perhaps more straightforward than it otherwise would have been. However, adapting the questionnaire to the German context still required a lot of consideration.</p>
<p>When writing your questionnaire, it is always good to start by looking in the literature for examples of how other people have asked similar questions. While we had to write some of the more unusual questions ourselves, we could borrow some language from other German-language surveys (like the <a href="https://www.diw.de/en/diw_01.c.615551.en/research_infrastructure__socio-economic_panel__soep.html">Socio-Economic Panel (SOEP)</a> or the <a href="https://www.gesis.org/en/allbus/allbus-home">German General Social Survey (ALLBUS)</a>) for more common demographic questions. This will also give you an idea of what scales are commonly used for your topic. It is also important to ensure that every question provides a complete range of mutually exclusive answer options, although you may occasionally want to constrain the options available. For example, we considered adding an “I don’t know” option when asking people to consider whether someone had a migration background, but we instead decided to force them to choose between “yes” and “no” because this is more useful for our purposes.</p>
<p>Another important consideration is the order of the questions and answer options. This may seem tedious, but order can have serious implications for the results of your study. In our case, we chose to ask questions about respondents’ immigration attitudes before showing them the treatment because we were concerned that the treatment could actually change someone’s attitudes in the short term. On the other hand, asking a question like this at the beginning of a survey might prime participants to think about immigration, which might not be ideal if you were trying to conceal the true intent of your study. These kind of effects are called “order effects” in survey terminology, and it’s worth carefully considering how they might affect your project <span class="citation">(Strack <a href="#ref-strack_order_1992" role="doc-biblioref">1992</a>)</span>. Overall, questionnaire design is an art and I cannot describe every step in detail here, so you should consult resources like the <a href="https://www.gesis.org/en/gesis-survey-guidelines/instruments/questionnaire-design">GESIS Survey Guidelines</a> for more information as you write your own.</p>
</div>
<div id="sampling-and-power" class="section level5">
<h5>Sampling and Power</h5>
<p>While you plan your research, you will also want to consider your target sample. Here, you not only need to decide on the population you hope to sample from, such as Germans without migration background (in our case), but also the sample size you need in order to make statistically sound claims. Calculating sample size may require you to balance many practical and theoretical considerations, as described in this <a href="https://psyarxiv.com/9d3yf/">recent paper</a> <span class="citation">(Lakens <a href="#ref-lakens_sample_2021" role="doc-biblioref">2021</a>)</span>. Some of these considerations might be how large of a sample you can afford to reach, effect sizes reported in previous research, and what the smallest substantively interesting effect size would be.</p>
<p>Ideally, you would determine your sample size with a power calculation, although these calculations can be complicated because of the many <a href="https://stats.oarc.ucla.edu/other/mult-pkg/seminars/intro-power/">factors</a> that influence power. Some calculations may require parameters like expected effect size that are difficult to estimate a priori, but pilot testing and previous research in your field can help you make an informed guess. I cannot cover all of the techniques you could use in such a short blog post, so you should explore previous work using similar designs and see how they have justified sample sizes for an idea of what methods might be most appropriate. When it comes to software, there are many options, fortunately including several that are free of charge. First, statistical software often includes built-in power commands, like “<a href="https://www.stata.com/manuals/pss-2power.pdf">power</a>” in Stata or “<a href="https://cran.r-project.org/web/packages/pwr/pwr.pdf">pwr</a>” in R. Beyond these options, there are free programs that were created specifically for power calculations, like <a href="https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower.html">G*Power</a> and <a href="https://sites.google.com/site/optimaldesignsoftware/home">Optimal Design</a>.</p>
<p>There are also alternatives to traditional methods of power calculation, which can be especially useful for complex experimental designs. Because we were conducting a replication and had access to the original data source, we used a simulation-based technique. We duplicated Abascal’s data to make a large population from which we then drew many samples, assuming that the effect size she finds is the true effect. For a power of 0.8, treatment effects in 80% of these samples should be significant. (For more information on this approach, see <span class="citation">Arnold et al. (<a href="#ref-arnold_simulation_2011" role="doc-biblioref">2011</a>)</span>.) A drawback of this method is that it required us to make assumptions about the effect size and variance based on U.S. data, which may not accurately predict responses in the German context.</p>
<p>Determining your target population, on the other hand, is mainly based on theoretical considerations. We pursued a sample that was representative of the adult German population without migration background in terms of gender, age, education, employment status, and region (East/West). To ensure balance on age and gender, we implemented quotas by gender and age, meaning that our sample would include a fixed number of men and women in each of three age groups. Note that your survey provider may be able to help you restrict your sample to certain demographic groups, but you may also need to implement screening questions to ensure the right sample composition. In our case, we had to screen for migration background and program quotas ourselves.</p>
<p>It is also important to note that despite our efforts to create a representative sample, samples drawn from online access panels may suffer from selection bias <span class="citation">(Bethlehem <a href="#ref-bethlehem_selection_2010" role="doc-biblioref">2010</a>)</span>. Participants must have Internet access, find the provider’s website, and opt into taking the survey, and people who do this may differ from those who do not on variables both observed and unobserved. While some have proposed methods like propensity scores to correct for this bias <span class="citation">(Schonlau et al. <a href="#ref-schonlau_selection_2009" role="doc-biblioref">2009</a>)</span>, this should be considered a weakness of online access panels, especially when extrapolating results to the general population.</p>
</div>
</div>
<div id="ethical-approval" class="section level3">
<h3>Ethical Approval</h3>
<p>Once you have generally decided on your research design and your survey population, it is a good idea to start your application for ethical approval.</p>
<p>Based on our experience, the application itself is generally not too time-consuming unless your design brings up many ethical issues, but the approval process can take several weeks or even months depending on your institution. It has historically taken us about 4-6 weeks to get approval at the University of Mannheim. You should definitely plan ahead, because it is critical that you have this approval before beginning the data collection. This is not only to ensure that you are acting within the ethical and legal guidelines of your institution and your field of research, but also because many journals require ethical approval for any work they publish.</p>
<p>When do you need to apply for ethical approval? Your institution’s rules may vary, but the <a href="https://www.uni-mannheim.de/en/about/organization/bodies-and-committees/committees-and-councils/ethics-committee/">University of Mannheim</a> requires it for any research on humans that:</p>
<ul>
<li>involves any personal or personally identifiable data</li>
<li>deceives subjects</li>
<li>involves psychological or physical health risks</li>
<li>triggers strong emotions or asks about traumatic experiences</li>
<li>manipulates subjects’ self-image</li>
<li>involves minors, or</li>
<li>presents risks to human dignity, life, health, and peaceful coexistence.</li>
</ul>
<p>In our case, the first two bullets applied. Any demographic information about a respondent counts as personal data, even if it could not be used to identify any specific person, so your research will almost certainly collect some personal data. Additionally, we employed deception by withholding the purpose of the study from respondents until the end of the survey, which we had to address by debriefing all participants after the survey was completed. If you are unsure whether your study requires ethical approval or not, it is a good idea to ask the staff person associated with your institution’s ethics committee. You should also look at their website, which might have additional information to help you plan your application. (Researchers in Mannheim can find an application checklist and other information <a href="https://www.uni-mannheim.de/en/about/organization/bodies-and-committees/committees-and-councils/ethics-committee/">here</a>.)</p>
<p>Your application should explain the ethical issues that will arise as part of conducting your research. You should explain why these features (like deception or data collection) are necessary for your design, and explain how you are minimizing any potential harms associated with them. You will also want to include as much information about your survey as you can, possibly including any planned debriefing text as well as the invitation to the survey, and maybe even the entire questionnaire depending on the expectations of your institution.</p>
<p>Another area that you may need to address in your application is data protection. This is perhaps especially important to researchers working in the European Union due to the <a href="https://gdpr-info.eu">General Data Protection Regulation (GDPR)</a> passed in 2016. I am not a lawyer and cannot provide a legal opinion on what regulations may apply to your research, but luckily the <a href="https://www.berd-nfdi.de">BERD @ NFDI</a> in Mannheim has put together a useful tool called the <a href="https://www.berd-nfdi.de/servicestools/legal-questions-in-data-science/">Interactive Virtual Assistant</a> that can give you more information on this topic (unfortunately only in German for now, but an English version is planned). In general, you should try to limit the amount of personal data you collect as much as possible, and make a plan to store this data securely. You will also want to avoid collecting any information that could be used to uniquely identify any of your respondents (or “personally identifiable information”, also called PII). Note that PII is not limited to names or addresses, but also includes combinations of variables that could identify a single person (such as a combination of ethnicity, age, and postal code). The ethics committee may be especially critical of designs that require the collection of such information.</p>
</div>
<div id="pre-registration" class="section level3">
<h3>Pre-Registration</h3>
<p>Once you have received ethical approval and have firmly decided on your design, you should consider pre-registering your research. A pre-registration is simply a report of your hypotheses, data source, and planned research design that you write and upload to a repository before starting data collection. This is meant to prevent practices like selective reporting and p-hacking, and to disclose confirmatory and exploratory analyses, which makes your decision-making more transparent to those who read and evaluate your work <span class="citation">(Nosek et al. <a href="#ref-nosek_preregistration_2018" role="doc-biblioref">2018</a>)</span>. This is certainly not a required step, but one that can help you plan and motivate your research, and that promotes principles of open science. One important thing to note is that pre-registering your design does not forbid you from performing exploratory analyses or even making design changes later on; it merely requires that you disclose that you have done so <span class="citation">(Simmons, Nelson, and Simonsohn <a href="#ref-simmons_pre-registration_2021" role="doc-biblioref">2021</a>)</span>.</p>
<p>If you would like to pre-register your study, you can find several useful templates on <a href="https://osf.io/zab38/">OSF</a>. A pre-registration will only require you to briefly describe your hypotheses, design, and planned analysis, information that you may have already collected for your ethics application. Once you have filled out your template, you can upload your pre-registration to a repository like <a href="https://osf.io/">OSF</a> or <a href="https://aspredicted.org/">AsPredicted</a> (among others), where it will be given a timestamp to verify when it was posted. Once posted, your pre-registration can be embargoed, making it invisible for several months while you complete your research. Note that you can also <a href="https://help.osf.io/hc/en-us/articles/360019930333-Create-a-View-only-Link-for-a-Project">create an anonymized link</a> to your pre-registration if you would like to include it in a manuscript submitted for peer review. If you are looking for an example, here is <a href="https://osf.io/2ygjd">our pre-registration</a>, although note that ours uses the “Replication Recipe” template designed specifically for replications.</p>
</div>
<div id="data-collection" class="section level3">
<h3>Data Collection</h3>
<div id="online-access-panels" class="section level5">
<h5>Online Access Panels</h5>
<p>You have planned your research, you received ethical approval, and you pre-registered your study – now you can finally begin collecting data. But we are still missing one key element: where is the data coming from? This is where an online access panel comes in. Online access panels have been used to study a variety of topics in the social sciences in recent years, from <a href="https://ejpr.onlinelibrary.wiley.com/doi/full/10.1111/1475-6765.12401">support for democracy</a> to <a href="https://journals.sagepub.com/doi/full/10.1177/0010414017740590">immigration attitudes</a>, among many other examples. They have also been a popular method of studying <a href="https://link.springer.com/article/10.1007/s11150-020-09529-4">reactions</a> to the COVID-19 pandemic, as they are relatively cheap, quick to deploy, and do not require in-person contact. Despite these benefits, there are also drawbacks that you should be aware of before collecting data. I have listed some of the main pros and cons below.</p>
<p>Pros:</p>
<ul>
<li>Fast data collection</li>
<li>Relatively low cost</li>
<li>Possibility of accessing a representative (non-student) sample</li>
<li>International data collection possible</li>
</ul>
<p>Cons:</p>
<ul>
<li>Only include internet users</li>
<li>Self-selection into participation can lead to biased population estimates <span class="citation">(Bethlehem <a href="#ref-bethlehem_selection_2010" role="doc-biblioref">2010</a>)</span></li>
<li>Some users may respond carelessly to get incentives</li>
<li>Potential loss of naivete (but not as much as MTurk, see <span class="citation">Chandler et al. (<a href="#ref-chandler_online_2019" role="doc-biblioref">2019</a>)</span>)</li>
<li>May be difficult to achieve large sample of minority groups</li>
</ul>
<p>If you decide that an online access panel might be a good fit for your research, your first step will be to compare prices and collect different offers.</p>
<p>Your institution will likely have rules about how many cost estimates you should collect (we needed three). You will want to contact several firms and provide them with information about your desired sample (especially size and any sample restrictions you want to impose), the timing you are hoping for, and the characteristics of your survey (programmed yourself or by the firm, mobile and/or web, time to complete). Make sure you request a large enough sample that you can run some pilot tests as well (more on this shortly). The providers you can choose from will depend on your desired population, but some of the larger firms in Germany include <a href="https://www.bilendi.de/">Bilendi/Respondi</a>, <a href="https://www.dynata.com/">Dynata</a>, <a href="https://www.kantarpublic.com/de">Kantar Public</a>, and <a href="https://yougov.de/">YouGov</a>. However, this is not a complete list, so you might consider asking colleagues or looking through the literature to see what providers might be a good fit for your planned sample. Throughout this process, you should also be in contact with your institution’s procurement department to make sure you are following all institutional rules. In our case, two providers sent us quotes and a third declined to bid on the project as they could not meet the specifications we asked for (this however still counted as a quote at our institution).</p>
</div>
<div id="programming" class="section level5">
<h5>Programming</h5>
<p>One decision you will have to make while getting quotes from survey companies is whether you plan to program your survey yourself, or whether you would like the company to program it for you. While it is certainly easier to “leave it to the experts”, programming the survey yourself is cheaper and gives you more control over the process, especially when it comes to implementing features like randomization. If you would like to try programming yourself, you might start by asking your institution whether they have a license for survey software. One of the most commonly used tools is <a href="https://www.qualtrics.com/">Qualtrics</a>, which requires little to no programming knowledge for simple setups, but is generally not free (except for surveys with <a href="https://www.qualtrics.com/free-account/?utm_lp=use-case-survey-software">fewer than 100 respondents</a>).</p>
<p>Other tools you may consider are <a href="https://www.otree.org/">oTree</a>, which is free and open source, and well-suited for interactive experiments and behavioral games (although more complicated to program), or <a href="https://www.unipark.com/en/survey-software/">EFS Survey Unipark</a>, a web-based service which is often available through German universities. No matter which software you choose, remember that the time and attention of your participants is valuable, and you should always strive to create an attractive and easy-to-use interface for your participants.</p>
</div>
<div id="pilot-testing-and-data-collection" class="section level5">
<h5>Pilot Testing and Data Collection</h5>
<p>Depending on the complexity of your design, you may also want to consider running some pilot tests before your main data collection to ensure that your survey works as intended. As one example, you might use a pilot test to identify any potential order effects by testing two versions of the survey where the questions are presented in a different order. In the case of our survey, we wanted to check a couple of things: first, that the treatment had an effect on ratings, and second, that the photos were perceived as expected (i.e. native German, migration background, or ambiguous). In our first pilot test, we found that many of our respondents did not understand the treatment, and that we needed to include more ambiguous photos. We then ran a second pilot test where we added a text description to the treatment graphs and required respondents to stay on the treatment page for one minute, which significantly improved respondents’ comprehension. Based on this experience, I would recommend that you plan for more pilot testing than you think you will need.</p>
<p>After our pilot tests, we started our data collection. It only took us about one week to collect just over 1,100 responses, although your experience may vary depending on the specific population you are sampling. Targeting smaller groups or older populations may take longer. Once we closed data collection, we also carefully cleaned the data. One drawback of online access panels is that some participants may speed through the survey in order to earn incentives. Some telltale signs of this are a very short completion time, “straightlining” responses (such as always choosing the first option), or nonsensical answers to open-ended questions. It is thus essential that you look through your data before analysis and screen out any cases that you think might be invalid. In our survey, we dropped 25 responses for a final sample size of 1,077.</p>
</div>
</div>
<div id="data-and-code-sharing" class="section level3">
<h3>Data and Code Sharing</h3>
<p>Once you have collected your data and finished your analysis, it’s time to share your results with the world! Of course you will write up your results in a manuscript, but you should also consider sharing your data and code with other researchers. Like pre-registration, this is optional, and this may not be feasible if your analysis relies heavily on proprietary data, but data sharing is becoming increasingly common in social science research, and may even be requested by some journals. Sharing your data and code fosters transparency and reproducibility, but there are also more self-interested reasons to share: uploading your work can open new opportunities for collaboration and generate more citations for your work.</p>
<p>When you are preparing to share your data and code, it is important to ask yourself: could another researcher reproduce your results without any additional information? In your dataset, this means ensuring clear coding and labeling, perhaps in a separate codebook file. It also means compiling your questionnaires so that people can see the source of the data (and translating them, if necessary). Perhaps most importantly, make sure that you anonymize or delete any personally identifiable information – refer back to your ethics application and be sure that you are doing what you promised to do. If your analysis includes other data sources that you do not own, you should also make sure to exclude these from your dataset (while informing users about where they can be accessed, and providing code that allows users to merge external data to your own). You can find more information on data preparation from <a href="https://www.icpsr.umich.edu/files/deposit/dataprep.pdf">ICPSR</a> and <a href="https://www.gesis.org/datenservices/daten-teilen/how-to-guide-daten-teilen">GESIS</a>. In your code, make sure you include labels to describe what each section does and that you reference all required packages. You should also have multiple people test your code, ideally someone who was not involved in the project, to ensure that everything is clear and that it works (ideally across different systems) as intended.</p>
<p>Once your code and data are replication-ready, you can upload them to a repository of your choice. You want to choose a repository that can ensure long-term preservation and a persistent identifier (such as a URL or DOI) so that people can find your information. Also make sure that your data will be accessible to other researchers, and if needed that it can be accessed anonymously during the peer review process. Some commonly used (and free!) repositories in the social sciences include <a href="https://osf.io/">OSF</a>, <a href="https://dataverse.harvard.edu/">Harvard Dataverse</a>, <a href="https://www.icpsr.umich.edu/web/pages/index.html">ICPSR</a>, and GESIS’ <a href="https://data.gesis.org/sharing/">SowiDataNet|datorium</a>.
You might also check with your university library, as some institutions may have their own repository (like Mannheim’s <a href="https://madoc.bib.uni-mannheim.de/">MADOC</a>).</p>
</div>
<div id="conclusion" class="section level3">
<h3>Conclusion</h3>
<p>If you have made it through all of these steps, congratulations on finishing your first data collection! As you can see, a lot of planning goes into any survey experiment. Despite this, I think it is worth the trouble. Collecting your own data can be a creative, rewarding, and even fun process that allows you to explore entirely new scientific questions. I hope this post has encouraged you to try it out for yourself, and I wish you the best of luck with your research.</p>
</div>
<div id="further-readings" class="section level3">
<h3>Further readings <a name="furtherreadings"></a></h3>
<ul>
<li>Mutz, D. C. (2011). <em>Population-based survey experiments</em>. Princeton University Press.</li>
<li>Auspurg, K., &amp; Hinz, T. (2014). <em>Factorial survey experiments (Vol. 175)</em>. Sage Publications.</li>
<li>Bansak, K., Hainmueller, J., Hopkins, D., &amp; Yamamoto, T. (2021). Conjoint Survey Experiments. In J. Druckman &amp; D. Green (Eds.), <em>Advances in Experimental Political Science</em> (pp. 19-41). Cambridge: Cambridge University Press.</li>
<li>Salganik, M. J. (2019). <a href="https://www.bitbybitbook.com/en/1st-ed/preface/"><em>Bit by bit: Social research in the digital age</em></a>. Princeton University Press.</li>
<li>Callegaro, M., Baker, R. P., Bethlehem, J., Göritz, A. S., Krosnick, J. A. &amp; Lavrakas, P. J. (2014). <em>Online Panel Research: A Data Quality Perspective</em>. Wiley.</li>
</ul>
<!-- Add something about the instructor -->
</div>
<div id="about-the-presenter" class="section level3">
<h3>About the presenter</h3>
<p>Joshua Hellyer <a href="mailto:joshua.hellyer@mzes.uni-mannheim.de"><i class="fa fa-envelope"></i> </a><a href="https://twitter.com/hellyer_josh"><i class="fa fa-twitter"></i></a> is a doctoral researcher at the Mannheim Centre for European Social Research (MZES). His research focuses on discrimination against ethnic and sexual minorities, particularly in the housing and labor markets.</p>
</div>
<div id="references" class="section level3 unnumbered">
<h3>References</h3>
<div id="refs" class="references">
<div id="ref-abascal_contraction_2020">
<p>Abascal, Maria. 2020. “Contraction as a Response to Group Threat: Demographic Decline and Whites’ Classification of People Who Are Ambiguously White.” <em>American Sociological Review</em> 85 (2): 298–322. <a href="https://doi.org/10.1177/0003122420905127">https://doi.org/10.1177/0003122420905127</a>.</p>
</div>
<div id="ref-arnold_simulation_2011">
<p>Arnold, Benjamin F., Daniel R. Hogan, John M. Colford, and Alan E. Hubbard. 2011. “Simulation Methods to Estimate Design Power: An Overview for Applied Research.” <em>BMC Medical Research Methodology</em> 11 (1): 94. <a href="https://doi.org/10.1186/1471-2288-11-94">https://doi.org/10.1186/1471-2288-11-94</a>.</p>
</div>
<div id="ref-bethlehem_selection_2010">
<p>Bethlehem, Jelke. 2010. “Selection Bias in Web Surveys.” <em>International Statistical Review</em> 78 (2): 161–88. <a href="https://doi.org/10.1111/j.1751-5823.2010.00112.x">https://doi.org/10.1111/j.1751-5823.2010.00112.x</a>.</p>
</div>
<div id="ref-chandler_online_2019">
<p>Chandler, Jesse, Cheskie Rosenzweig, Aaron J. Moss, Jonathan Robinson, and Leib Litman. 2019. “Online Panels in Social Science Research: Expanding Sampling Methods Beyond Mechanical Turk.” <em>Behavior Research Methods</em> 51 (5): 2022–38. <a href="https://doi.org/10.3758/s13428-019-01273-7">https://doi.org/10.3758/s13428-019-01273-7</a>.</p>
</div>
<div id="ref-gereke_demographic_2022">
<p>Gereke, Johanna, Joshua Hellyer, Jan Behnert, Saskia Exner, Alexander Herbel, Felix Jäger, Dean Lajic, et al. 2022. “Demographic Change and Group Boundaries in Germany: The Effect of Projected Demographic Decline on Perceptions of Who Has a Migration Background.” <em>Sociological Science</em> 9.</p>
</div>
<div id="ref-lakens_sample_2021">
<p>Lakens, Daniël. 2021. “Sample Size Justification.” PsyArXiv. <a href="https://doi.org/10.31234/osf.io/9d3yf">https://doi.org/10.31234/osf.io/9d3yf</a>.</p>
</div>
<div id="ref-nosek_preregistration_2018">
<p>Nosek, Brian A., Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. 2018. “The Preregistration Revolution.” <em>Proceedings of the National Academy of Sciences</em> 115 (11): 2600–2606. <a href="https://doi.org/10.1073/pnas.1708274114">https://doi.org/10.1073/pnas.1708274114</a>.</p>
</div>
<div id="ref-schonlau_selection_2009">
<p>Schonlau, Matthias, Arthur van Soest, Arie Kapteyn, and Mick Couper. 2009. “Selection Bias in Web Surveys and the Use of Propensity Scores.” <em>Sociological Methods &amp; Research</em> 37 (3): 291–318. <a href="https://doi.org/10.1177/0049124108327128">https://doi.org/10.1177/0049124108327128</a>.</p>
</div>
<div id="ref-simmons_pre-registration_2021">
<p>Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2021. “Pre-Registration: Why and How.” <em>Journal of Consumer Psychology</em> 31 (1): 151–62. <a href="https://doi.org/10.1002/jcpy.1208">https://doi.org/10.1002/jcpy.1208</a>.</p>
</div>
<div id="ref-strack_order_1992">
<p>Strack, Fritz. 1992. “‘Order Effects’ in Survey Research: Activation and Information Functions of Preceding Questions.” In <em>Context Effects in Social and Psychological Research</em>, edited by Norbert Schwarz and Seymour Sudman, 23–34. New York, NY: Springer New York. <a href="https://doi.org/10.1007/978-1-4612-2848-6_3">https://doi.org/10.1007/978-1-4612-2848-6_3</a>.</p>
</div>
</div>
</div>
]]>
      </description>
    </item>
    
  </channel>
</rss>