Survey, Field, and Ad Campaign-Level Experimental Tests of Political Persuasion

• In Event: Experiments in Representation

Abstract

The study of the persuasive effects of digital advertising in political campaigns is beset by research design challenges. Forced-exposure survey experiments may overstate the effect of campaigns when fully-treated subjects’ outcomes are measured immediately post-treatment. Field experiments may understate the effect of advertisements when subjects are assigned to a small dose in the context of a full campaign (and may fail to comply with such assignments). Even so, we employ both designs. We measure the effects of 46 campaign advertisements using survey experiments on Facebook and Mechanical Turk and 66 advertisements using state-of-the-art digital advertising field experiments on the Facebook platform. Consistent with much of the previous literature, we find small, mostly homogeneous effects that often cannot be distinguished from zero. To resolve this methodological knot, we held out a control group of 370 thousand voters from a massive 3.5-million-person digital advertising campaign targeted at swing voters in battleground states. The full weight of a 10-month-long, pro-Biden, anti-Trump Facebook advertising avalanche caused no change in early voting among Biden leaners but decreased early voting among Trump leaners by 0.59 percentage points (difference-in-CATEs p = 0.0005). This suite of experiments sheds light on longstanding theoretical debates over heterogeneity, durability, and dosage in the study of campaign persuasion.

Authors

• Minali Aggarwal, Yale University

• Jennifer N Allen, Massachusetts Institute of Technology

• James Barnes

• Alexander Coppock, Yale University

• Dan Frankowski

• Tara McGowan, ACRONYM

• Solomon Messing, Twitter

• Kelly Zhang, Massachusetts Institute of Technology