Role of Instrument Design in Framing the Validity Argument for a Set of Mathematics Assessments

• In Event: Informing Perspectives on Validation: Narratives From Mathematics Education Scholarship

Abstract

, the focus is on the design part of the measurement, that is, the conceptual, theoretical and empirical work that is needed before any interpretations can be made, and that is also the background framework to the argument for the different strands of validity evidence.
To illustrate this, we will use the context of an assessment system based on a novel mathematics curriculum (Assessing Data Modeling) developed using the BEAR Assessment System. This system has been constructed with an explicit focus on a learning progression for how students in the later elementary and middle school years learn statistical reasoning. It consists (after a number of iterations of structure) of six dimensions of learning, each formed as a sequence of increasingly sophisticated levels of student understanding. As the work progressed, assessment, instruction, and learning were increasingly intertwined as we produced artifacts that mediated relations among professional development, learning and assessment. The talk will follow the story of the development of this structure, and the ways that theoretical underpinnings and empirical information about validity has been used in a formative way to shape that structure. Then eventually, the same sorts of information are then used summatively in the validity arguments. This two-step usage for the forms of validity evidence is crucial to the success of the eventual measurements, and is an important feature of the enlarged focus.

Authors

• Mark R. Wilson, University of California - Berkeley

• Richard Lehrer, Vanderbilt University