How Powerful is the Evidence in Criminology? On Whether We Should Fear a Coming Crisis of Confidence

• In Event: Design Challenges in Quantitative Research

Abstract

A crisis of confidence has struck scientific disciplines like psychology and neuroscience and that crisis is intimately tied to the lack of statistical power in many of the studies in those areas. Our concern is that criminology might be similarly at risk. Thus, our primary objective is to examine carefully the level of statistical power that characterizes studies produced in criminology. To address this issue, we gathered 321 average effect size estimates and 271 average sample size estimates from 80 published meta-analyses. We then provide the very first estimates of the positive predictive value (PPV), which yields an estimate of the probability that any given study reaches an accurate conclusion by rejecting the null hypothesis. Our analyses indicate that, on average, criminological studies are characterized by a relatively moderate level of statistical power and a moderate PPV. Yet, a sizable number of studies exhibited relatively high (low) levels of power and high (low) PPV values. Thus, substantively important variation was observed.

Authors

• J.C. Barnes, University of Cincinnati

• Michael F TenEyck, University of Texas at Arlington

• Travis Pratt, University of Cincinnati