The Origins of Statistical Significance Testing in Medical Research

• In Event: Reproducibility and Other Problems: Practical and Institutional Responses to Contemporary Crises in Science - I

Abstract

Statistical significance testing is fundamental to medical research, so much so that research findings and “facts” must be adjudicated by these routine statistical calculations. However, despite its widespread use, significance testing has been the subject of debate and criticism for decades, and no one seems content with how researchers use and interpret significance tests. This puts modern significance testers in the awkward position of recognizing epistemological shortcomings of significance testing, at least as it is used in practice, while simultaneously bowing to its epistemological authority. Significance testing is so engrained in scientific practice that most researchers take it for granted and do not consider why their disciplines rely on it.

Looking at historical publications, including textbooks and research articles archived on JSTOR, I trace the origins of significance testing in medical research in order to understand what motivated researchers to adopt, promote, and retain it over time, despite alerts about its shortcomings. In so doing, I treat significance testing as a signal detecting machine. Considering significance testing to be a machine broadens our conceptual understanding of technology/technique. Unlike most machines studied in STS, significance testing is not tangible; it is built upon mathematical proofs, symbolic meanings, and rhetoric, which makes it relatively open to many different (mis)applications and (mis)interpretations and debates about what is “legitimate” use. By understanding when it was adopted and how it was used in different periods, I provide insight into medical researchers’ interests and motivations for using this flexible, contentious, and powerful research tool.

Author

• Christopher J Holmes, University of Wisconsin - Madison