Ecologically Plausible? Comparing the Independent and Paired Samples t-Test With Nonrandom Missingness and Skewed Data

• In Event: Effect Sizes, Multilevel Models, and Causal inference

Abstract

Missing data often presents difficulties for educators seeking to establish the potency of an intervention using a pre-test/post-test design. These problems are exacerbated in settings when high student turnover seriously depletes sample size and reduces power. Previous research has shown that the independent samples t-test outperforms the paired-samples t-test when there is high missingness and a low correlation between pre-test and post-test scores (Márquez-Muñoz, 2019; Sperling, Zwahr-Castro & Walls, 2017). This study demonstrates that the independent samples t-test maintains its advantage over the paired samples t-test when data are skewed, which often happens when a large proportion of students do either very well or very poorly on an assessment. Implications for educators in districts with high student mobility are discussed.

Authors

• Sequoia Naomi Callahan, Texas A&M University - College Station

• Rick Sperling, St. Mary's University San Antonio

• Bridget Horta, Saint Mary's University of San Antonio