Simulation Study: The Effects of Missing Data on the Partial Credit Trees Differential Item Functioning Detection Performance

• In Event: Important Considerations for Survey Data

Abstract

This study examines the interactive effects between missing data and complex DIF/DSF patterns on the performance of the Partial Credit Trees (PCTrees) DIF method (semi-parametric DIF detection through model-based recursive partitioning). A simulation study was conducted in which the factors manipulated were missingness mechanisms, missing proportions, DIF/DSF patterns, types of DIF covariates, and cross-group examinee ability distribution differences. Preliminary results showed that (1) with list-wise deletion, no significant differences appeared in the Type 1 error rates (all very close to the 5% nominal level) across all missingness mechanisms and DIF/DSF patterns; and (2) PCTrees power rates were most strongly affected by non-constant DIF/DSF patterns, multinomial covariate, the presence of cross-group examinee ability distribution differences, MCAR mechanism, and increasing missing proportions.

Authors

• Chunling Niu, University of Incarnate Word

• Nathaniel P Wilson, University of Kentucky

• Kelly D. Bradley, University of Kentucky

• Rui Jin