Hybrid Models for DIF Detection: A Simulation Study of Classical and Machine Learning Approaches

• In Event: AERA Poster Session Wednesday 1:45pm
  In Poster Session: Poster Session II -- AERA Division D Section 1: Measurement, Psychometrics, and Assessment

Abstract

This simulation study evaluates traditional, machine learning, and hybrid methods for detecting Differential Item Functioning (DIF) in dichotomous test items. Using the 3PL IRT model, datasets were generated under varying test lengths, sample sizes, and DIF types. Logistic regression and IRT residuals effectively detected uniform DIF in larger samples but showed limited power for non-uniform DIF. LASSO regression was highly dependent on regularization strength, underperforming in small samples and overflagging items with relaxed penalties. Random Forests consistently outperformed other methods, identifying the highest number of true DIFs across diverse conditions. Ensemble approaches such as majority voting and hierarchical pipelines further enhanced accuracy. Findings support the use of machine learning and hybrid strategies to ensure fairness in educational and psychological testing.

Authors

• Mohammad Jahanaray, Virginia Commonwealth University

• Atena Pasha, University of South Carolina