Panacea or Poison: Can Propensity Score Modeling Replicate the Results from Randomized Control Trials? – PART I

• In Event: Advances in Evaluation Methods

Abstract

Regardless of institutional focus (e.g., law enforcement, corrections), identifying the practices that will best reduce crime and victimization requires ample research with strong methodology. Understood as the “gold standard” in research design, the randomized control trial (RCT) has been shown to provide reliable and valid findings. However, RCTs are often not feasible in many criminal justice settings because of ethical and practical considerations. As a substitute for RCTs, researchers have increasingly had to rely on other quasi-experimental designs and statistical techniques. One of the more popular techniques is propensity score modeling (PSM), which is designed to simulate the effects of an RCT. With the growing popularity and technological ease of using PSM, this research project funded by the National Institute of Justice addresses the critical question: Can PSM methods replicate the results from RCTs? Using a selection of RCT databases available in the National Archive of Criminal Justice Data (NACJD), this study tests commonly used PSM techniques to examine how close related effect sizes actually are to that of an RCT in criminal justice settings. As Part I of this two-part presentation, we explain the process of testing such techniques and discuss the findings from one of the tests.

Authors

• Christopher Michael Campbell, Portland State University

• Ryan M Labrecque, Portland State University