A Meta-Review: The Gap Between Simulated and Real Data in CDM Research

• In Event: AERA Virtual Poster Session 8
  In Poster Session: Poster Session 1

Abstract

This study conducts a meta-review of studies on cognitive diagnostic modeling (CDM) to enlighten discrepancy in the simulated and real data. The studies on CDM have used simulated data that are generated from particular parametric assumptions. The real assessments used as empirical examples in these studies however were not initially designed for cognitive diagnosis. In this study, we investigate over 20 published articles and examine reproducibility and likelihood of real data given the parametric settings assumed in the simulation. The outcomes indicate discernible gap between the simulated and real data distributions. Based on our empirical analysis, we can give consideration to the need of likelihood of real and simulated data for more information of how CDMs perform in real-world assessments.

Authors

• Sook Hyun Park, The University of Texas at Austin

• Hyeon-Ah Kang, The University of Texas at Austin