Item Response Theory Models for Learning With Item-Specific Learning Parameters

• In Event: Advancing Diagnostic Psychometric Models to Enhance Student Learning: Statistical Theory, Methods, and Applications

Abstract

Differentiated instruction is common in the modern classroom and has been shown to be beneficial for both remedial and advanced students. Popular online learning platforms such as IXL, Khan Academy, and Dream Box can provide extra practice and immediate feedback. These online learning platforms provide differentiated instruction to students by providing them with practice and interventions targeted to their current ability level, which can be assessed dynamically.
Several IRT models for learning are proposed, in which items or testlets have their own learning parameters that may be estimated from data. The purpose of the simulation study is to see how well the learning parameters may be estimated under different models. The IRT models are coupled with a linear logistic test measurement model that provides greater parsimony when item difficulties can be explained by characteristics that can be coded in a Q-matrix. A two-stage estimation procedure is developed that helps avoid a potential identifiability problem. Then, we assess the finite sample properties in simulation under a variety of conditions. A separate simulation study is then conducted to examine the gains in learning efficiency that a learning model may afford while paired with an adaptive item selection algorithm. Finally, we fit several learning models to a real dataset that involves an intervention to help students learn spatial rotation skills, and discuss issues such as model fit and model selection. Learning parameter values were fixed at the same values for all of the learning models, ranging from 0.4 to 2.0. In the simpler learning models, learning parameters were much more accurately estimated using their posterior means. For an exam with 50 items and 2000 examinees, the mean absolute deviations between the posterior mean and true learning parameter values were 0.0673 for a simple linear growth model and 0.2433 for the most complex model, respectively. The adaptive item simulation quantifies the improved benefit of selecting items tailored to an examinee’s ability versus two other cases: random item selection and selecting items with the largest learning parameter value. Regardless of the model used, adaptive item selection consistently led to larger improvements in the examinees’ ability parameter compared to the other two methods.
An analysis was performed on a real dataset of spatial rotation data collected by the University of Illinois Department of Psychology. This was a 50-question exam with a sample size of 350 based on the Purdue Spatial Visualization Test. Learning model parameter estimates were made using both the 1PL and the LLTM as the true model. Estimates were shown to be widely different between the two models. This was likely because the question difficulties in this dataset are not well-approximated with the LLTM. It was also possible that fatigue and lack of motivation were responsible for an observed plateau in examinees’ performance after the second block of items. Due to the need for shorter exams, adaptive item selection would be a good choice. We can see that when these models hold, it gives a large advantage over other methods of item selection.

Authors

• Albert Yu, University of Illinois at Urbana-Champaign
  Presenting Author

• Jeff Douglas, University of Illinois at Urbana-Champaign
  Non-Presenting Author