Quota Sampling Using Facebook Advertisements

• In Event: Poster Session: Political Methodology

Abstract

Researchers in different social science disciplines have successfully used Facebook to recruit subjects for their studies. Such convenience samples, however, are not representative of the general population. Here we develop and validate a new method to recruit respondents using Facebook ads and quota sampling. To test the method, we used ads to quota sample 2,432 US respondents for a survey on climate change opinion. Simultaneously, we conducted a nationally representative survey asking identical questions using a high-quality probability sampled online panel survey. Results from the Facebook-sampled survey are similar to those from the online panel survey; furthermore, results from the Facebook-sampled survey approximate results from the American Community Survey (ACS) for a set of validation questions. These findings suggest that using Facebook to recruit respondents is a viable option for survey researchers wishing to approximate population norms.

Survey researchers have increasingly turned towards web-based survey tools like Amazon's Mechanical Turk and Google Consumer Surveys as cost-efficient methods to survey diverse target populations. An emerging literature within psychology and public health have also used Facebook to recruit respondents into research studies. We extend this emerging literature that leverages Facebook as a sampling platform to test whether Facebook advertisement-based recruitment can efficiently generate public opinion estimates that approximate results from a nationally representative sample. Our method exploits the recently updated Facebook Marketing API to semi-automate a quota sampling process. Through a proof-of-concept study on US climate opinions, we demonstrate that researchers can cost-effectively recruit respondents through quota sampling using Facebook ads. As part of this project, we plan to release an R package that automates the quota sampling process on Facebook.

To demonstrate the validity of the quota sampling method described above, we conducted a study to compare the results of a quota-sampled Facebook survey with a high quality probability-sampled online panel survey (GfK). The raw, unweighted demographic summary statistics and survey responses of the two surveys are similar. The results become even more similar after we weight them to the March 2016 Current Population Survey using inverse propensity score weighting. Furthermore, results from the Facebook-sampled survey approximate demographic summary statistics from the American Community Survey (ACS).

Authors

• Baobao Zhang, Yale University

• Matto Mildenberger, University of California Santa Barbara