Panacea or Poison: Can Propensity Score Modeling Replicate the Results from Randomized Control Trials? – PART II

• In Event: Advances in Evaluation Methods

Abstract

Regardless of institutional focus (e.g., law enforcement, corrections), identifying the practices that will best reduce crime and victimization requires ample research with strong methodology. Understood as the “gold standard” in research design, the randomized control trial (RCT) has been shown to provide reliable and valid findings. However, RCTs are often not feasible in many criminal justice settings because of ethical and practical considerations. As a substitute for RCTs, researchers have increasingly had to rely on other quasi-experimental designs and statistical techniques. One of the more popular techniques is propensity score modeling (PSM), which is designed to simulate the effects of an RCT. With the growing popularity and technological ease of using PSM, this research project funded by the National Institute of Justice addresses the critical question: Can PSM methods replicate the results from RCTs? Using a selection of RCT databases available in the National Archive of Criminal Justice Data (NACJD), this study tests the reliability and validity of commonly used PSM techniques by comparing accompanied effect sizes to those of RCTs in criminal justice settings. In Part II of this two-part presentation, we meta-analyze the differences found in the effect sizes generated through the PSM and RCT methodologies.

Authors

• Ryan M Labrecque, Portland State University

• Christopher Michael Campbell, Portland State University