Investigating Middle School Students' Grasp of Evidence

• In Event: 66.027 - Ask for Evidence: Identifying, Preventing, and Counteracting Scientific and Educational Myths

Abstract

Objectives and theoretical framework. The evidence that students encounter in school tends to be epistemically simple, whereas real evidence encountered in the world varies greatly in amount, comprehensiveness, scope, methodological quality, complexity, interpretive distance, and so on (Authors, in press; Samarapungavan, in press). Students need to learn how to handle much more complex forms of evidence.
We have developed a theoretical framework called “Grasp of Evidence” (Authors, in press) for analyzing evidential reasoning. The notion of grasp of evidence builds on Ford’s (2008) discussion of grasp of practice, involving both constructing and critiquing claims. We examine grasp of practice as comprising five dimensions: (1) evidence analysis; (2) evidence evaluation; (3) evidence interpretation; (4) evidence integration; and (5) laypersons’ use of evidence. Each of these dimensions is further unpacked in terms of the three central components of the AIR model of epistemic cognition: (A) Aims and value; (B) Epistemic Ideals or criteria and (C) Reliable epistemic processes. We use this framework to address the question: What are the strengths and weaknesses of seventh grade science students’ grasp of evidence along each of these five dimensions?
Method. We implemented a one-year study involving over 500 seventh graders in 20 science classes taught by 5 teachers who collaborated on the design of all materials. Students regularly engaged in constructing, revising, and evaluating models based on evidence. The evidence varied broadly in quality, interpretive affordances, and complexity so as to simulate a broad range of evidence found in real settings. Our research team videorecorded two classes taught by each teacher throughout this time. Our data sources are transcripts and written work from throughout this implementation. We developed coding schemes for assessing oral discourse and written work along the five dimensions of the Grasp-of-Evidence framework.
Results. Our results highlight strengths and weaknesses along each of the five dimensions. Here are several selected highlights: Evidence evaluation. Students learned to evaluate many dimensions of evidence, including sample size, use of proper comparisons, and careful methods of observation, but they did not understand deeply the reasons for these principles, so that they misunderstood that studies with small samples can sometimes be valuable. Evidence interpretation. Students generally evaluated the relevance of evidence effectively, but with insufficient attention to the diagnosticity of evidence in distinguishing between alternative models. Laypersons’ evaluation of evidence. Most students came to understand that anecdotal sources were of low quality, but their understanding of what conferred genuine expertise was shallow.
In full results to be presented, we will provide quantitative data on ideals and reliable processes found for each dimension. For example, in the evidence evaluation dimension, a few of the many reliable processes referenced included using appropriate observational methods (34%), using proper controls (14%), tight inferencing (8%), and corroboration by others (6%).
Significance. These findings can guide instructional efforts to promote better reasoning about evidence. We are re-designing instruction to capitalize on students’ strengths and to afford opportunities to think in new ways about evidence where weaknesses were found.

Authors

• Clark A. Chinn, Rutgers University

• Ravit Golan Duncan, Rutgers University

• Sarit Barzilai, University of Haifa