Self-Report and Process Data Indicators of Engagement: Unique Predictors of Performance Across Assessment Types

• In Event: Validity of Insights Derived From Learning Technology Log Data

Abstract

We examine indicators of student engagement extracted from process data from an online assessment system as predictors of student learning. Student learning was assessed via four distinct learning outcomes varying according to stakes (low- v. high-stakes) and span (one-time v. aggregated). Participants included high school students enrolled in an AP Statistics course (N = 320, Mage = 16.76 years; 60.2% female). While largely uncorrelated with self-report measures, certain process data variables had significant relations with learning outcomes. For example, frequency of score report checking, though uncorrelated with self-reported engagement, was associated with all learning outcomes. These findings suggest process data from online assessment systems can help broaden and deepen our understanding of student behavior above and beyond self-report.

Authors

• Teresa M. Ober, University of Notre Dame

• Maxwell Hong, Meta

• Daniella Rebouças, Houghton Mifflin Harcourt Publishing

• Matthew Carter, University of Notre Dame

• Cheng Liu, University of Notre Dame

• Ying Cheng, University of Notre Dame