Using Evidence-Centered Design to Broaden the Range of Cognitive Performances in College Tests

• In Event: Understanding the Cognitive Processes Tapped by Complex Assessments

Abstract

This paper will present results from validity studies that illustrate the value of employing evidence-centered design (ECD) to develop a new assessment approach for post-secondary domain-specific learning. The prototype assessment focuses on community college programs in biology and economics. Analysis focuses on data from expert panels and test results from 296 students (148 biology, 148 economics) stratified by how many courses they had taken. The prototype assessments featured 8 task scenarios (4 biology, 4 economics), comprising 41 biology items (32 open-ended) and 47 economics items (24 open-ended). Results indicate the ECD design process produced instructionally sensitive assessments that measure a wider than usual range of cognitive performance in college biology and economics, including skills for everyday knowledge application.

Authors

• Louise G. Yarnall, SRI International

• Larry Gallagher, SRI International

• Geneva D. Haertel, SRI International