I Never Meta-Analysis Like You Before: Voter Mobilization by Electoral Salience

• In Event: The Campaign, Issues and Salience

Abstract

There is an inherent tension between political science research practice and the practical application of our work: when experiments are run vs. when scholars (and practitioners) are most interested in applying the findings. Researchers often seek to conduct mobilization field experiments in low-salience elections (e.g., local elections or primaries) because smaller sample sizes are needed to provide adequate statistical power to detect an effect, and any mobilization tactic may be more likely to cut through the noise. However, the findings of our work are often applied to high-salience elections (midterm and presidential general elections), where the effects are attenuated by the heightened attention and crowded communication landscape.

This paper presents a series of meta-analyses of field experiments about common voter mobilization tactics segmented by whether the experiment was conducted in a high- or low-salience election. Statistically, tests for heterogeneity indicate that high- and low-salience should not be pooled. Substantively, the differences between mean estimated effect for each and every voter mobilization tactic make clear that our understanding of mobilization and voting behavior is distorted if we do not separate experiments by election context.

Our meta-analyses calculate that common mobilization tactics are in low-salience elections have larger impacts than in high salience elections by anywhere from 50% (door-to-door canvassing) to 59% (volunteer phone calls) to 78% (social pressure direct mail) to 132% (SMS text-messages). Moreover, the different magnitudes in high- and low-salience electoral contexts creates a different picture of the relative efficacy of these tactics – with important implications for understanding voting behavior and campaign activity.

Further refinement of our operationalization of electoral salience will (hopefully) yield additional insights about how mobilization tactics are conditioned by political context. The broad category of low-salience elections (primaries, municipal and state) shows considerable signs of non-random heterogeneous effects across experiments about which we are gathering more data.

Although often conflated, election salience is distinct from individual level turnout probability (e.g., Arceneaux & Nickerson, 2009) in conditioning treatment effects. Treatment responsiveness peaks for individuals with a 50-50 likelihood in the election wherein the experiment is conducted. Election salience conditions the height of that peak.

Taken as a whole, our results offer guidance about inference from the corpus of voter mobilization experiments and a note of caution about applying the results. Researchers conducting experiments should be clear about what they can contribute to understanding of voting behavior when conducting experiments in particular contexts. Experiments in high- and low-salience elections make distinct contributions and move understanding forward in different ways. Scholars drawing on experiments for broader understanding of voting behavior should be mindful of the generalizability of findings across electoral contexts. Scholars communicating with practitioners should use an effect estimate appropriate for the electoral context in which voters are being mobilized. We also encourage researchers to turn their focus to specific voting sub-populations (minorities, low-income areas, young voters) to improve meta-analytic estimates of tactics directed at these groups in both high- and low-salience elections.

Authors

• Christopher B Mann, Skidmore College

• Katherine Haenschen, Northeastern University