Teaching Statistical and Substantive Significance Under a True Null Hypothesis

• In Event: Technology and Tools Useful for Teaching Research Methods and Statistics

Abstract

Modern, massive digital data requires computer-intensive algorithms (data science) for analysis. However, small data sets continue to be analyzed with classical, inferential statistical methods. Regrettably, these methods have been tainted by the abuse, misuse, and misunderstanding of statistical significance. Understanding statistical significance requires an appreciation of theoretical sampling distributions of summary statistics under a true null hypothesis. Here, computer-simulated sampling distributions of Pearson’s correlation coefficients and p-values reveal that statistical significance with small sample sizes filters out effect size errors that would otherwise be considered substantively significant under a true null hypothesis. My programs and datasets will be available on Figshare.com (a public registry), and the results can be reproduced and replicated using free SAS software accessible on the internet.

Author

• Eugene Komaroff, Keiser University