Rater Cognition in Teachers: Applying the Rater Cognition Model in a Large-Scale Assessment System

• In Event: Novel Approaches to Large-Scale Assessment Data Analysis and Practice

Abstract

Previous research has profiled raters’ scoring patterns in terms of the outcome of their scores, profiling raters as “severe” or “lenient” scorers (e.g., Eckes, 2012). Fewer studies have attempted to profile raters in how they reach their score decisions, focusing on what cognitive processes or lack of certain cognitive processes are present during the scoring process. As the use of constructed response items in large scale assessments continues to grow, it is important to gain a comprehensive view of how raters approach the scoring process. The current study attempts to understand the unique experiences teachers bring to the scoring process, and offers suggestions for rater training as it relates to teacher involvement in the scoring process.

Authors

• Elizabeth Leighton, University of South Carolina

• Robert L. Johnson, University of South Carolina

• Stefanie S. Sebok-Syer, Queen's University - Kingston

• Ashlee A. Lewis, University of South Carolina

• Edward Wolfe