How Data Comes to Be

• In Event: Emotions, Beliefs, and Political Behavior in Pluralistic Societies

Abstract

The collection and consumption of information that is used to measure and judge substantive concepts involves several different actors, all of whom come from diverse backgrounds and disciplines. Importantly, different actors' theoretical interpretation of substantive concepts may, and often do, differ. In some cases these differences can be beneficial, leading otherwise dispersed insights to aggregate into something closer to the true estimate. However, in other cases where strategic considerations matter, such differences can undermine the ultimate goal of empirical inquiry by producing a biased estimate. We conceptualize the collection and categorization of information as a process involving distinct stages, and develop a new theory that focuses on the actors who participate at each stage. The process starts with the documentation/observation stage, which represents the original transcription or tabulation of information into primary sources. The second stage is the measurement phase, where information from primary sources is used to construct empirical measures. The last stage is the consumption stage where empirical measures are used. By analyzing this sequence of stages, we show how differences of opinion regarding a substantive concept, and the knowledge of these differences, create incentives that undermine the relationship between substantive concepts and the empirical measures meant to represent those concepts. Our approach allows us to formalize the construct validity of empirical measures at different stages and assess what problems arise from strategic issues in the empirical process.

Authors

• Christopher J. Fariss, University of Michigan

• Scott Tyson, University of Rochester