The good, and the bad, and the ugly: selecting control variables when testing criminological theories

• In Event: Theory-testing and relevant methodologies in criminological research

Abstract

This paper demonstrates how adjusting for the wrong control variables can be deleterious for theory-testing using observational data. Several criminological theories are founded on the premise that certain variables are causally related to an outcome. For example, deterrence theory posits that certainty of punishment prevents criminal behaviour, social disorganisation theory sustains that neighbourhood disorder is linked to higher crime rates, and procedural justice theory premises that legitimacy beliefs can foster public compliance with the law. Due to common ethical and logistical impracticalities of experimental designs, empirical research aiming to assess the empirical validity of causal relationships is often limited to observational data, and in most applications, not even quasi-experimental designs are realistically viable. In such circumstances, criminologists usually rely on a selection-on-observables approach and estimate statistical models that adjust for several control variables. However, selection of controls is often overlooked. With the goal of adjusting for confounding variables that block all backdoor paths between the treatment and the outcome and compute unbiased estimates, past research has often also inadvertently adjusted for “bad controls”. In this paper, we explore how adjusting for bad controls can introduce collider bias, mediation bias, overcontrol bias, bias amplification, among other issues. Drawing on graphical tools that permit the visualisation of causal paths, we first use simulation to assess the degree to which adjusting for bad controls introduces bias. Then, we replicate results of three criminological studies that might have incurred in collider bias, mediation bias, and overcontrol bias. We conclude with a discussion on the importance of thorough considerations of theory in empirical research. Given that the data generating process is unknown in observational studies, theoretical premises should inform the empirical strategy, including the selection of control variables. Without careful consideration, statistical models are bound to fail in their theory-testing goals.

Authors

• Nicolas Trajtenberg, University of Manchester

• Thiago Oliveira, University of Manchester