Machine Learning in Large-Scale Educational Assessment: A Systematic Review of Methods, Rigor, and Model Performance.

• In Event: Extending the General Linear Model: From Classical Regression to Bayesian Estimation and Machine Learning

Abstract

Large-scale educational assessments are high-dimensional datasets that capture cognitive outcomes and contextual factors across multiple educational levels. The complexity and scale of these datasets have led to machine learning (ML) adoption for predictive modeling and analysis of LSA data. We followed PRISMA guidelines and identified 71 peer-reviewed studies that met our inclusionary criteria of ML adoption for LSA research within the last 10 years. Our preliminary result shows Random Forest is the most common ML algorithm, and accuracy across different models ranges from 72.59% to 95.7%. Also, the methodological rigor of ML implementation varied substantially, with few studies addressing class imbalance, incorporating complex survey weights, and utilizing cross-validation techniques. We present the ML methods, gaps, and suggest analytical best practices.

Authors

• Oluwaseun Peter Farotimi, University of Tennessee

• Comfort Happiness Omonkhodion, University of Central Florida

• Funke Anuoluwapo Dada, Georgia Southern University

• Seun Ajoseh, University of Florida

• Debbie L. Hahs-Vaughn, University of Central Florida

• Haiyan Bai, University of Central Florida