Validity of Automated Scoring in Game-Based Assessment

• In Event: 21.058 - Special Topics in Test Validity Research: Artificial Intelligence in Test Construction and Scoring

Abstract

The SimScientists Games project, funded by the National Science Foundation, is developing rigorous assessments of cognitively-principled science games that use automated scoring to assess key science ideas and practices in two ecosystems games for middle school science that engage students in challenging and fun deep science learning. This paper will describe how SimScientists Games are designed and validated as formative assessment resources for reinforcing and extending middle school concepts and science practices and collaboration skills, using established methods of gathering validity evidence used in prior SimScientists simulation-based assessment projects: alignment with national standards for science, expert review, analyses of students thinking-aloud, and analyses of teacher and student data gathered from classroom field testing.

Authors

• Matt Silberglitt, WestEd

• Kim Luttgen, WestEd

• Melissa Rego, WestEd

• Charlie Mahoney, WestEd

• Andrew Grillo-Hill, WestEd

• Daniel Brenner, WestEd