Evidence Model: Evaluate Complex Problem-Solving Competency From Process Data

• In Event: The Use of Process Data in Educational Assessment

Abstract

Computer-based assessments allow practitioners to collect process data by logging students’ interactions with assessment tasks, and promise to furnish more evidence to support claims about what a student knows and can do. However, culling evidence from messy log files is a challenging task. Guided by the evidence-centered design framework, this study examines process data in the context of the 2012 PISA complex problem-solving assessment, and aims to answer two research questions: (a) Can student process data reflect distinct cognitive competencies/skills? (b) To what extent can ratings of individual student performance be supported by evidence from process data? The study will demonstrate how an evidence model can be established through a theory-driven approach for analyzing process data.

Author

• Zhitong Yang, Educational Testing Service