Extending the Additive Factors Model to Assess Student Learning Rates

• In Event: Cognitive Models for Assessment

Abstract

The increasing use of educational technologies like Intelligent Tutoring Systems (VanLehn, 2006) and educational games in classrooms is producing vast amounts of process data that capture rich information about the learning process as it unfolds. These data can be used to provide dynamic assessments of student knowledge and progress as learning occurs, allowing us to move beyond static assessments of student achievement at a single point in time. To model these process data, Cen, Junker, and Koedinger (2006) proposed the Additive Factors Model, which extends the traditional Rasch model from Item-Response Theory to accommodate both a cognitive model representation of the domain and changes in student knowledge across learning. Originally, the Additive Factors Model was used to improve and refine the cognitive models used to drive adaptive problem selection in Intelligent Tutors. Recently, we have been using and extending the model to assess student knowledge and learning. We applied the Additive Factors Model to several educational datasets from Intelligent Tutoring Systems, spanning domains such as Geometry, Algebra, and English Grammar. The model’s estimates of student ability were consistently and significantly correlated with traditional post-test outcomes. Furthermore, we used the model’s predictions to classify students into three groups: those who improved less with each learning opportunity than the model predicts, those who improved more per opportunity than the model predicts, and those who progressed roughly at the predicted rate of improvement. Based on this method, we found that students’ “learning rate” group significantly predicted their gains from pre-test to post-test. Students who improved less per opportunity than the model predicts exhibited the smallest pre- to post-test gains, whereas students who improved more per opportunity than the model predicts exhibited the biggest gains. This suggests that we may be able to compare real student performance to the Additive Factors Model’s predictions, in real time as learning unfolds, to assess students’ learning rates and target individualized instructional policies accordingly

Authors

• Ran Liu, Carnegie Mellon University

• Kenneth R. Koedinger, Carnegie Mellon University