Assignments as a Source of Error in Critical-Thinking Assessment

• In Event: Measuring College-Level Skills

Abstract

Inconsistency in scores across multiple tasks is an important source of measurement error within assessment processes. The present study investigated the magnitude of this variance and whether this source of error was a function of assignment characteristics. A total of 343 student papers were nested within 18 assignments sampled across the institution. Critical thinking scores were provided by trained faculty members who rated each paper using a common rubric. Hierarchical linear modeling indicated that 18% of the total variation can be attributed to differences in average critical thinking scores between each assignment. Approximately 47% of this variance was accounted for by the extent to which each assignment allowed for students to express their own perspective.

Authors

• John David Hathcoat, James Madison University

• Jeremy Penn, Oklahoma State University