Knowledge Creation and Use in Assessment and Evaluation of Classroom-Based Formative Assessment Tools in Mathematics

• In Event: From the Common Core State Standards to Rigorous Instructional Practice: Creating Knowledge Around the Development and Use of Formative Assessment Tools

Abstract

Perspective and Objectives

This presentation describes the role of rigorous assessment and evaluation in strategy development and refinement. It uses as an example the Gates Foundation’s investment in formative assessment, termed the Mathematics Design Collaborative (MDC), to help teachers incorporate the CCSS’ rigorous content and problem solving goals in mathematics.

CCSS ontologies and the validity argument for MDC assessments were two key intellectual foundations for validating MDC. The ontologies graphically display the CCSS progressions and provide tools for mapping alignment, monitoring student progress and connecting a range of resources to CCSS (Baker & Chung, 2009). The validity argument (Herman & Choi, 2010) lays out the claims that link the MDC assessments to the purposes they are intended to serve (AERA, AERA, NCME, 1999; Kane, 2006). The argument also establishes a developmental cycle of evidence collection and analysis. Ontologies of content and cognitive demand have been used to conduct alignment studies, using a new, Foundation-supported technology tool. Pilot test data have informed MDC improvement, and, as part of field testing, data on reliability of scoring, scores and use are being collected.

Methodology

As the intervention moves to implementation, a rigorous evaluation is being planned, in collaboration with Foundation partners. Based on the theory of action, the evaluation features a quasi-experimental design and specially developed measures of fidelity of implementation and learning outcomes. These new measures are essential for sensitivity to MDC intent, are linked to the ontologies, and are being examined following in line with the project’s validity argument.

Field test sites for the intervention are serving as validation sites for the new instruments, so that assessment and evaluation strategies are developing hand in hand with MDC roll out. The overlap also serves to maximize efficiencies in development and to minimize time to scale up, but requires close coordination and cooperation among researcher partners and school sites and compromise in balancing the needs of research and the response burden for practitioners.

Significance

The development, validation and evaluation of the Foundation’s investment demonstrate the range of expertise involved in bringing knowledge to practice. Assessment design, development and validation have brought together researchers, practitioners and experts in mathematics, mathematics education, learning, assessment development, measurement, computer science, and graphics design. Evaluation design broadens even further the required pool of expertise, e.g., qualitative and quantitative methodology, professional development, change, school and district policies and practice. The efforts also well demonstrate the speed with which new ideas must be brought to practice to keep up with the CCSS schedule and the need for and development of new knowledge creation and sharing tools in evaluation and assessment to promote the CCSS transformation.

Authors

• Joan L. Herman, University of California - Los Angeles

• Eva L. Baker, University of California - Los Angeles