Incorporating Multiple Data Source to Explore Group Strategy Variations

Session Type: Coordinated Paper Session

Abstract

This coordinated session focuses on incorporating multiple data source to explore the group strategy variations beyond merely response data to address potential measurement fairness issues. Our session covers a broad range of process data exploration and showcases both advanced psychometric modeling and data-driven approach to identify the variations in problem-solving strategies and provide potential solution to leverage these variations in the measurement. The first paper presents a response time-based finite-mixture item response theory approach for flexible modeling examinee-and-item-specific item-solving strategies. The second paper introduces a mixture hidden Markov model to leverage action sequences with elapsed time to highlight test takers’ strategy variation in reading pause states. The third paper demonstrates how to combine the Seldonian algorithm and adversarial neural networks to estimate the degree that students’ online or remote learning preparedness levels impact feature extraction and predictive model interpretation, focusing on the “time-on-task” features. The fourth paper presents the strategy variations in missing responses and behavioral pattern extraction in interactive computational thinking tasks across countries via a sequence mining approach. These studies lead a pressing direction of integrating multiple data source in large-scale assessments to provide a new angle to assure the equitable measurement across groups with various backgrounds.

Sub Unit

• Individual Papers, Innovation Demonstrations, Coordinated Sessions, or Organized Discussions

Session Organizer

• Qiwei He, GEORGETOWN UNIVERSITY

Individual Presentations

• A Response Time-based Finite-mixture Item Response Theory Model for Item-Solving Strategy - Sijia Huang, INDIANA UNIVERSITY BLOOMINGTON; JINWEN Luo, UCLA; Minjeong Jeon, UCLA

• Identifying Group Difference by Latent Sequence Modeling on Process Data with Elapsed Time - Ling Chen, Columbia University; Qiwei He, GEORGETOWN UNIVERSITY

• Towards Fair Learning Analytics: An Application of the Seldonian Algorithm - Okan Bulut, University of Alberta; Jinnie Shin; Wallace Nascimento Pinto, University of Florida

• Profiling Behavioral Patterns in Interactive Computational Thinking Tasks Across Countries - Qiwei He, GEORGETOWN UNIVERSITY; Eugenio Gonzalez, EDUCATIONAL TESTING SERVICE

Discussant

• Steven Culpepper, UNIVERSITY OF ILLINOIS AT URBA