Assessment in the Advent of Process Data and AI

Session Type: Invited Speaker Session

Abstract

The purpose of this session is to share recent work at the intersection of assessment, cognition, and computing. Three papers demonstrate how the analysis of response process data gathered in computer-based testing environments can be used in the design, evaluation, interpretation, and improvement of student assessments. As a complement to this, the fourth paper uses principles of cognitive assessment and psychometrics to inform the evaluation of natural language generation models.

Sub Unit

• SIG-Cognition and Assessment

Chair

• Jimmy de la Torre, University of Hong Kong

Papers

• Profiling Behavioral Patterns in Interactive Computational Thinking Tasks Across Countries: Sequence Mining With Process Data - Qiwei He, Georgetown University

• Evaluating Examinee Engagement Through Modeling the Relation Between Response Time and Item Word Count - Pooya Razavi, Edmentum; Sonya J. Powers, Edmentum

• Sequential Mastery Detection of Multiple Attributes With Response Accuracy and Response Time Data - Sangbeak Ye, Florida Atlantic University

• Evaluating Natural Language Generation Model Evaluation: A Measurement Theory Perspective - Susu Zhang, University of Illinois at Urbana-Champaign; Ziang Xiao, Johns Hopkins University; Vivian Lai, Visa Research; Vera Q. Liao, Microsoft Research

Discussant

• Steven Andrew Culpepper, University of Illinois at Urbana-Champaign