Modeling Longitudinal Multi-Rater Data

• In Event: Advances in Expert-Coded Data

Abstract

Numerous applications of latent variable models in the social sciences use Bayesian techniques and dynamic prior specifications to help overcome issues with bridging latent variable estimates across time periods (e.g., Martin & Quinn 2002, Schnakenberg & Fariss 2014). Such prior specifications rely on theory to make restrictive assumptions about the dynamics of the latent trait in question. Further, it is critical to align prior assumptions with the data generating process, to avoid low accuracy and over-confidence in estimates. Using the Varieties of Democracy Project as an example, I argue that commonly used dynamic latent variable models provide a poor theoretical and empirical fit to the concepts that many panel surveys, especially in the social sciences, seek to measure. In particular, I argue that political institutions rarely follow smooth dynamic processes, but rather exhibit periods of stasis, or slow drift, followed by sudden change. But researchers in other fields, and even commercial data scientists, can also benefit from careful theorizing about dynamics. For example, restaurant quality changes over time in a manner surprisingly consistent with institutions: quality tends to hold reasonably constant, or drift slowly, except when changes in management or other shocks cause large alterations to operations. I argue that change-point models (Chib 1998, Park 2011) provide a better theoretical match to institutional (and restaurant quality) dynamics and show that latent variable models built upon a theory of institutional change-points better fit such data than do standard models of smooth dynamic evolution. I also demonstrate a series of robustness tests to check dynamic modeling assumptions, discuss the broad costs and benefits of relying on dynamic modeling assumptions when fitting latent variable models to panel data, and situate my contribution relative to recent work on robust dynamic latent variable models (Reuning, Kenwick & Fariss 2019).

Author

• Daniel Pemstein, North Dakota State University