Bottom-Up Innovation: Engaging Practitioners in Designing Performance Assessments at the High School Level

• In Event: The Challenges and Possibilities for Instructionally Valuable Performance Assessments in High-Stakes Environments

Abstract

Purpose and Significance
The high-stakes policy environment in which assessments have been used over the last decade have led to a heavy reliance on standardized selected-response tests designed and implemented by large-scale testing companies. Such tests have been criticized as being too far away from teachers' curriculum and instruction. On the other hand, teachers have been designing summative assessments for generations, with test content closely tied to the content and skills taught. Teacher designed tests have been criticized as being too idiosyncratic, unreliable, and lacking comparability across teachers to be useful as credible measures. This project deliberately sought to engage teachers in the process of designing summative performance assessments aligned to the Common Core Literacy Standards, with the goal of combining both testing paradigms - designing assessments that are close to teachers' classroom instruction and curriculum, but with a common set of criteria for quality and a process for vetting quality. This paper reports on the results of engaging grade 9-12 English teachers in the design process with experts. The findings have important implications for the possibility of incorporating teacher-designed curriculum-embedded performance assessments as a valid measure of student learning in which teachers' work "counts for something".

Theoretical Framework
In previous work that the authors completed for the New York City Department of Education, a "top-down" approach in which experts authored performance tasks for a task bank was used. The approach used in this project was the opposite - a participatory approach that is "bottom-up." The theory of action underlying the strategy of involving teachers as Design Team members, in a context in which these assessments may potentially be used for high stakes purposes, is that such involvement will 1) increase the validity of the assessments, 2) support user buy-in, 3) build expertise in the design and use of assessments, and 4) make the assessments more instructionally useful. This paper focuses on whether the involvement of teachers in the assessment design process supports the first three goals.

Methods and Sources
This paper draws on the experiences of the assessment experts who were involved in facilitating professional development and design studios with the 9-12 "Design Team" participants in the project that is a focus of this symposium. It draws,as well on relevant prior experiences in a two-year project within the same district using a different assessment design approach.

Findings
While the involvement of teachers appears to have built some buy-in into the Task Design and Implementation Guidelines as well as the common scoring rubric for evaluating the student writing products, there remained some skepticism about the validity of the assessments that teachers had built in relation to their proposed use - for the evaluation of teachers' contribution to student performance. Teachers involved in the 9-12 Design Team gained important foundational knowledge about the features of high quality designs, but insufficient time to work with the team resulted in few of them being equipped to independently develop high quality writing tasks, without ongoing expert supervision and review.

Authors

• Ruth Chung Wei, Stanford University

• Justin Wells

• Susan E. Schultz, Stanford University

• Daisy A. Martin, Stanford University