How Experiments Help Campaigns Persuade Voters: Evidence from 617 Ad Experiments

• In Event: Voting, Political News and the Campaign

Abstract

Television ads capture the lion's share of expenditures by American political campaigns. Are some of these ads more effective than others? If so, why, and with what implications? In this paper, we analyze evidence from an archive of 146 real studies conducted by campaigns in 2018 and 2020 using a platform called Swayable. Across these experiments, 51 campaigns analyzed the effects of 617 unique advertisements intended to persuade voters to vote for a candidate. Analyzing the responses of over 500,000 survey responses across these experiments, we demonstrate two major empirical results about the variation in persuasive effects. First, we show that ad effects are tightly distributed around a small, positive mean. For example, the effects of presidential ads on vote choice in 2020 have a mean of 0.85 percentage points and a standard deviation of 0.34---great ads are only so much more effective than mediocre ads. Second, we show that theories about what makes political advertising more or less effective like ad tone or messenger have very limited explanatory power and appear highly context dependent. These two findings imply that experimentation compounds the influence of money in elections. Conducting randomized experiments can help campaigns estimate which of a set of particular advertisements persuades best, but the benefits of this procedure accrue mainly to campaigns well-financed enough to act on their findings at scale.

Authors

• Luke Hewitt

• David Broockman, Stanford

• Ben Tappin

• Alexander Coppock, Yale University