Effect of Missing Data in Computerized Adaptive Testing on Accuracy of Item Parameter Estimation: A Comparison of NWEA (Northwest Evaluation Association) and WINSTEPS Item Parameter Calibration Procedures

• In Event: Roundtable Session 26
  In Roundtable Session: Rasch Measurement SIG Roundtable Session

Abstract

The purposes of study are to investigate effect of missing data in computerized adaptive testing (CAT) on accuracy of item parameter estimation based on Rasch model using Northwest Evaluation Association (NWEA) and WINSTEPS calibration methods, and to provides information on the relationship between two methods. The results illustrate that the test length (or missing rate) and calibration methods have not only a statistically significant impact, but also practical implication on the accuracy of recovery of item parameters. The relationship between true and estimate item parameters from CAT data with large missing rate show that item parameter recovery is not as good as conventional thinking by using WINSTEPS, while the robustness of NWEA calibration methods for CAT data is encouraging.

Authors

• Shudong Wang, NWEA

• Gregg Harris, Northwest Evaluation Association