Establishing Statistical Significance at Scale for Pattern-Based Items

• In Event: Psychometrics and Educational Measurement Poster Session 2

Abstract

Tests of statistical significance often play a decisive role in establishing the empirical warrant of evidence-based research in education. The results from pattern-based assessment items are categorical and multimodal and do not immediately support the use of measures of central tendency as typically related to interpretations of measures of statistical significance. Using results from the implementation of pattern-based items (PBIs) at scale (n = 421,000) as part of the statewide Interim Assessment Program (IAP) in Texas, this paper is meant to address how statistical significance can be established across scale. We illustrate the use of either a Fisher’s Exact Test or a Pearson's Chi-squared Test to establish the level of statistical significance for differences in results for pattern-based items.

Authors

• Walter M. Stroup, University of Massachusetts Dartmouth

• Anthony J. Petrosino, Southern Methodist University