Evaluation of the Alignment Between CBAL Periodic Accountability Assessments and the Common Core State Standards: A Confirmatory Study

• In Event: Building and Validating Cognitively Based, Technology-Enhanced English Language Arts Assessments

Abstract

The purpose of this study was to collect evidence from panels of teachers on the perceived alignment between a subset of the Mathematics and ELA grades 6-8 Common Core State Standards (CCSSs) and a subset of questions from the CBAL Periodic Accountability Assessments (PAAs) thought to align with those CCSS. CBAL questions are embedded in complex, contextualized tasks, where a task/question may measure more than one standard. This elevates the complexity of an alignment study over that which is typically encountered, where a traditional multiple-choice item is intended to measure one standard (Tannenbaum, Baron, & Kannan, 2013).
The current study is confirmatory; CBAL researchers and test specialists performed an initial alignment analysis, mapping specific questions from available PAA tasks to the grade 6-8 CCSS. The result of the mapping work was a sparse matrix of questions by standards.
In this study, two independent panels of teachers, one for mathematics (N = 11) and one for ELA (N = 12) participated in an alignment process modeled after procedures recommended by Webb (1999; 2007). Panelists evaluated the extent of alignment between questions and standards (defined by the sparse matrix) on two criteria: content overlap and cognitive complexity overlap. Analyses identified points of agreement and disagreement with the purported alignment.
The results supported the purported alignment. The teacher panels confirmed that most questions initially judged to map to selected standards were aligned. For mathematics, 24 of 27 standards (89%) were confirmed to align in content, and 18 of 27 standards (67%) were confirmed to align in cognitive complexity. For ELA, 31 of the 35 standards (89%) were confirmed as aligned for content and for cognitive complexity. Webb (2007), citing Subkoviak (1988), suggests six questions as a guideline for considering when a standard is adequately covered. Using this measure, 11 mathematics standards and six ELA standards were confirmed as mapping to the CCSS.
Alignment results serve as a source of [content-based] validity evidence (Martone & Sireci, 2009). The results also provide formative feedback to test developers on where to fortify intended alignment to targeted standards. The methods and procedures and complete results will be presented, and the challenges of conducting alignment in the context of complex, innovative task types will be discussed.

Authors

• Patricia Baron, ETS

• Richard J. Tannenbaum, ETS