Post‑Hoc Bayesian Hypothesis Testing for Improving Inference in Underpowered Educational Research Designs

• In Event: Innovations for Complexity in Models and Design: Inference, Estimation, Classification, and Longitudinal Issues

Abstract

This proposal introduces a post‑hoc Bayesian Hypothesis Testing (BHT) method designed for educational research contexts where studies often deviate from ideal sampling plans. The BHT approach rescales prior probabilities for the alternative hypothesis using a logistic function of the observed‑to‑expected sample size ratio, penalizing underpowered studies and rewarding design fidelity. It also applies effect size shrinkage to correct for Type M error. A comprehensive Monte Carlo simulation demonstrates that BHT discounts evidence from small‑N designs, transitions sharply as planned sample size is approached, and converges with traditional inference when designs are adequately powered. This method offers a computationally accessible bridge between NHST and fully Bayesian analysis, improving transparency, interpretability, and evidence quality in field‑based educational research.

Authors

• M. Shane Tutwiler, University of Rhode Island

• Szu-Yu Zoe Kao, University of Rhode Island