Assessing Scientific Argumentation by Middle School Pupils and Testing a Learning Progression for Argumentation

• In Event: Assessment for Educational Accountability in History and Science: Beyond Recall

Abstract

Current educational policy seeks to develop the higher order skills of reasoning (analysis, synthesis, evaluation) (Commission of the European Communities, 2007). Moreover, within both national and international frameworks there is an increasing emphasis on the ability to construct and critique scientific arguments (Mullis et al., 2009; National Research Council, 2011). However, to date, summative assessment in science has been dominated by lower order skills – in particular recall (National Research Council, 2001; Sunal & Wright, 2006). Assessing students’ facility with argumentation through the use of pencil and paper items has been an underexplored area of research. While considerable effort has been devoted to developing analytical schemes for the analysis of oral discourse, little research exists that has attempted to measure students’ individual capabilities with written argumentation and critique. Moreover, a singular focus on testing students’ scientific concepts fails to develop students’ ability to reason critically (Kuhn et al. 2000; Sandoval, 2003) leaving them with a naïve view of scientific epistemology (Driver et al., 1996; Lederman, 1992) and a limited view of science.

In this paper, we report on a study conducted within the San Francisco Unified School District (SFUSD) in collaboration with the Berkeley Evaluation and Assessment Research (BEAR) Center, the SERP Institute, and 7 SFUSD teachers. The focus of this work has been on assessing grade 8 students’ facility with argumentation about of changes of state and density. The aim has been to test a learning progression of students’ ability to engage in domain-specific reasoning. To test our model, we have written a range of assessment items and scoring guides. The items have been administered to a sample of 500 grade 8 students of all abilities. In this paper we present a mixed-methods analysis of student responses and argumentation offering:

• The theoretical foundations of our argumentation progress map and examples of argumentation questions. Additionally, we explore the issues raised by the design of such questions.
• A description of the iterative process of co-development with SFUSD teachers to improve their quality and establish the validity of these items.
• A quantitative analysis of the written responses and a qualitative analysis of a set of think-aloud student responses.
• An analysis based on Item-Response Theory (IRT) that provides information on how well the model embedded in the argumentation progress map matches the empirical data using Rasch modeling and Wright maps.
• An illustration of the many difficulties encountered in producing valid assessment items.

Results show that students find engaging in critique harder than constructing scientific explanations. Whether this is because of any intrinsic cognitive demand of critique, or whether it is because the lack of emphasis on the critical evaluation in their science education to date, along with the curricular implications, will be discussed.

Authors

• Jonathan F. Osborne, Stanford University

• Bryan Henderson, Stanford University

• Anna MacPherson, Stanford University

• Evan Szu, Stanford University