The Psychological Reality of the Learned "p < 0.05" Barrier

• In Event: Motivation in Mathematics

Abstract

Traditional statistics instruction emphasizes a 0.05 significance level for inferential tests. Here, we investigate the consequences of this training for researchers’ mental representations of probabilities – whether 0.05 becomes a boundary, i.e., a discontinuity of the mental number line. 25 graduate students with statistical training viewed pairs of p-values and judged whether they were “similar” or “different”. As predicted, participants were more likely and faster to judge p-values as “different” when they crossed the 0.05 boundary (e.g., 0.049 – 0.051) compared to when they did not (e.g., 0.019 – 0.021). This categorical perception effect suggests that traditional statistical instruction creates a psychologically real chasm between significant and non-significant p-values. This distortion is undesirable given modern approaches to statistical reasoning that de-emphasize dichotomizing p-values.

Authors

• Vimal V Rao, University of Minnesota - Twin Cities

• Jeffrey K. Bye, University of Minnesota - Twin Cities

• Sashank Varma, Georgia Tech