Why RCTs Fail? A New Mechanical Response to Stevenson

• In Event: Advances in New Mechanistic Criminology: Contending with Complexity in the Social World

Abstract

The limits of criminological interventions assessed through rigorous evaluation (e.g., randomized controlled trials) have increasingly come under question. Recent work highlights that over 50 years of rigorous evaluation has demonstrated little to no impact (Stevenson, 2023). Such works argue that, among other reasons, the weak impacts are observed in interventions evaluated via rigorous evaluation because they suffer from restricted scope and are subject to the complexities of the social world. These complexities are often not measurable or missed due to the adoption of an engineering approach to science that eschews theory’s role in scientific explanation. This paper contends the weak impacts of criminological interventions are due to avoiding theory and overlooking critical mechanistic fragility–the highly conditional nature of complex pathways that allow for desired outcomes only within a precise range of conditions. To explore this, we engage in a negative case analysis of a Western state’s swift-certain-fair parole intervention (N = 3,812). We find the intervention’s negative cases are not reflective of random error, but instead highly structured in terms of their etiological, contextual, and constitutive causes. We contend the use of New Mechanical criminology can guide rigorous evaluation to transcend their current limitations associated with the engineering view.

Authors

• Christopher M. Campbell, Portland State University

• K. Ryan Proctor, AIMS/USAFA

• Richard E. Niemeyer, USAFA/WERC