The Importance of Normative Models in Research on Voter Rationality

• In Event: Experimental Research on Voter Rationality, Engagement, and Turnout

Abstract

Ideally, experimental research on political behavior and political psychology is concerned with questions that are of both scientific interest, in addressing a clear research gap, and of normative interest, in saying something important about the functioning of democracy and to what extent citizens act as competent, responsible democrats. Examples of this is research on political knowledge (Carpini and Keeter, 1996), research on elite cues (Druckman, Peterson and Sluthuus, 2013; Barber and Pope, 2019; Broockman and Butler, 2017), and research on how citizens process political information (Coppock, Leeper and Mullinix, 2018; Kahan et al., 2012, 2017; Taber and Lodge, 2006; Guay and Johnston, 2021). Oftentimes the normative conclusions are very bleak: citizens are uniformed (Carpini and Keeter, 1996), follow elite cues mindlessly (Barber and Pope, 2019; Broockman and Butler, 2017), and process political information in a biased, self-serving manner (Baekgaard et al., 2019; Glinitzer, Gummer and Wagner, 2021; Kahan et al., 2012, 2017; Taber and Lodge, 2006; Guay and Johnston, 2021; Stanley et al., 2020).

However, it is not always clear such pessimistic conclusions are warranted. For instance, (Lupia, 2006) criticizes what he refers to as the "elitist move" in research on political knowledge. This occurs when researchers measure political knowledge in terms of questions about information that academics, journalists, and political pundits typically value (e.g. "who is the head of the Federal Reserve?"). The elitist move is assuming that such questions have similar value to citizens with societal responsibilities often are different from those of the elites. The failure, according to Lupia (2006), lies in not explicitly discussing what constitutes sufficient knowledge to be able to act as a democratically competent citizen. We believe a similar problem is ubiquitous in much experimental research on political behavior and political psychology.

As argued by Hahn and Harris (2014), it is instructive to review the notion of bias in statistics when thinking about respondent bias. An estimator is said to be "biased" when the expected value of the estimator deviates from the true population parameter (μ): bias(estimator) = E[estimator]−μ. Bias should thus be thought of as systematic deviations from something. When political scientists talk about partisan bias, confirmation bias, biased information processing, biased assimilation, this "something" is often not specified. Rather, it is taken as more or less self-evident that certain behaviors or patterns of responding are "problematic". In his critical discussion on mainstream political knowledge research Lupia (2006) argues that many commonly used questions in fact measure "superfluous knowledge", i.e. knowledge that is not necessary to act as a democratically competent citizen. To reach this conclusion the author first provides a definition of "competence" (in the context of democratic behavior) and derives what should constitute sufficient political knowledge. This is hence the author’s preferred normative model of political knowledge. This also makes it clear that showing that voters fail to answer superfluous knowledge questions is not strong evidence of incompetence, since this is not necessarily a deviation from the normative model.

By explicitly describing his preferred normative model Lupia (2006) makes the logic of inference transparent; the type of results that would prompt him to raise concerns about voter incompetence can be clearly distinguished from those that would not. This also makes it possible to criticize his normative point of departure. Perhaps a different model would be preferable, possibly one where voters ought
to know the name of the head of the Federal Reserve?

Most experimental studies seem to have a clear view of what constitutes "undesirable" behavior by the respondents. However, many studies do not specify what would constitute "desirable" behavior and what assumptions are needed to draw the inference that respondents engage in behavior that is not desirable. In short, how *should* respondents behave if they were unbiased? We refer to this as an implicit normative model since important information needed to draw inferences is left out. We argue that this is a problem in much experimental research that seriously undermines our ability to evaluate the evidence put forward and that makes the logic of inference opaque.

In this paper we review several common experimental research designs and show that researchers using these designs often implicitly rely on questionable assumptions when drawing inferences about rationality and bias among respondents. We provide empirical evidence showing that many reported results are compatible with several vastly different models of voter behavior. We end by discussing how researchers can make their experiments more informative and their logic of inference more transparent.

Authors

• Mattias Agerberg, University of Gothenburg

• Love Christensen, University of Gothenburg