Investigating the Portability of Predictive Models Across Diverse Courses

• In Event: Saturday Roundtable 7:45 am
  In Roundtable Session: Examination of Individual and Contextual-Level Variables in Large-Scale Assessments (Table 1)

Abstract

This study evaluates transfer learning models' efficacy in predicting student outcomes across diverse courses. Three machine learning algorithms were trained on individual course data and tested on all other courses. Results revealed significant variation in model performance, highlighting the difficulty of creating generalized predictive models from learning management system (LMS) data. We analyzed course similarity's impact on model portability with regression analysis showing it had a greater effect on specificity compared to courses from the same department. Additionally, the proportion of students passing in both training and testing courses, along with the size of the training dataset, emerged as key predictors of the model's performance. These findings shed light on the challenges and factors influencing predictive success in this context.

Authors

• Joseph Olson, University of Alabama - Birmingham

• Zhen Xu, Teachers College, Columbia University

• Nicole Pochinki, Teachers College, Columbia University

• Renzhe Yu, Teachers College, Columbia University