Formative Assessment Practices: Construct Definition and Measurement Challenges

• In Event: Genres of Formative Assessment

Abstract

Objectives
Just as effective formative assessment practice starts with clear articulation of learning goals and the pathways through which those goals may be accomplished, so too must research on formative assessment start with clear conception of target constructs and valid measures through which to understand them. The paper offers a framework for inquiry and uses data from two projects to illustrate options and challenges for valid research measures.
Theoretical Framework
Widely conceptualized as a process used during instruction, formative assessment is viewed as one of most effective interventions for improving learning (CCSSO, 2008; Black & Wiliam, 1998). Recent studies, however, have revealed wide variation in observed effects and suggest the need for more nuanced views that include critical variables in practice (E.g., , Furtak et al. 2008; Hattie and Timperley 2007; Kingston and Nash 2011; Heritage et al., 2009; Herman et al., 2006; Herman, Osmundson & Silver 2010)
At the same time, cognitive psychology provides a strong theory base supporting the instrumental learning value that assessment instruments can add. Research has long demonstrated a testing effect: testing increases retention (Izawa, 1970; Donaldson, 1971, Bartlett and Tulving, 1974); repeated testing boosts the effect (Roedigner and Karpicke, 2006); and constructed response tests produce higher learning gains than selected response ones (Kang, McDermott & Roedigner, 2007; McDaniel et al., 2006). Clearly as well, formative assessment as a process cannot work well without valid evidence of learning.

Methods and Data Sources
Data from two projects are used to illustrate tools for instantiating the model and investigating the impact of core variables in practice. The first project is a three-state RCT, involving 170+ teachers, of the effects of adding curriculum embedded-formative assessments to a hands-on, elementary school science curriculum. The study used observation, interview, and log data to assess the quality of teachers’ formative assessment processes and multiple measures of student outcomes. Process measures looked at the same core dimensions of practice: communication of goals, assessment and analysis strategies, feedback and use of results.
The second project, a statewide study of teachers’ use of formative assessment, collected and analyzed teachers’ self-identified formative assessment “tools” in more than 100 schools. Rubric dimensions included clarity of goals, alignment with standards, intellectual challenge, accessibility, format, grain size, frequency and diagnostic value.
Results
Study 1 results demonstrate the value of triangulation, the difficulty of capturing quality in large scale implementation measures, the differential sensitivity of various measures of implementation and outcome. Study 2 results show wide variation in how teachers interpret formative assessment and uneven quality.

Scientific Significance
Measurement has been deemed the Achilles heel of educational research. This presentation shares measurement tools and raises questions to strengthen the sensitivity of research on formative assessment. By suggesting strengths and weaknesses in current teacher practice, the presentation also carries implications for research, policy, and practice.

Authors

• Joan L. Herman, University of California - Los Angeles

• Christine Ong, University of California - Los Angeles