Epistemic Criteria and Reliable Processes as Indicators of Argument Quality in Science Students' Argumentation

• In Event: Advancing Students' Argumentation: Moving Beyond Structure

Abstract

Objectives and theoretical framework: The endeavor to promote argumentative competence cannot be achieved without a comprehensive and productive understanding of what counts as a good argument. In this presentation we apply a novel approach for analyzing the quality of argumentation in students’ written and oral discourses that broadens the predominantly structural analyses that are prevalent in the field. Our framework is based on the AIR model of epistemic cognition and practices (Authors, 2014). Structural approaches such as Toulmin’s Argumentation Pattern (Toulmin, 1958) can help identify some weaknesses in students’ arguments, but structural analyses alone cannot explain why, for example, a structurally complex argument against anthropomorphic global warming is a poor argument. Our method makes use of two elements of the AIR model--epistemic ideals or criteria and reliable epistemic processes--to evaluate argument quality. A structurally rich anti-global-warming argument is poor because it fails to meet epistemic criteria such as fit with multiple lines of evidence and because it draws on unreliable processes for generating knowledge claims, such as cherry picking evidence or utilizing poor, unsystematic methods for combining evidence.

Data sources and methods: We illustrate the usefulness of our approach to analyzing argument quality though an analysis of data from our project, Promoting Reasoning and Conceptual Change In Science (PRACCIS). PRACCIS is one-year long life science curriculum contextualized in model-based inquiry majorly for the 7th graders. The written data is drawn from 332 students in a lower performing, highly diverse school. Students in the written task evaluated three pieces of evidence bearing on two possible models and wrote reasons to support the model they chose. The analysis of oral argumentation comes from oral discussions in which the 7th graders developed a food web on the basis of four pieces of evidence varying in quality.

Selected results: Our analysis showed that 7th graders develop arguments that vary not only in structural quality but, orthogonally, in quality according to our analysis of epistemic criteria and reliable processes relied on to make the arguments. High-quality arguments invoked epistemic criteria such as relying on a preponderance of evidence, consistency with other domain theories, and a preference for direct versus indirect evidence. High quality arguments also evinced use of sophisticated analyses of observational processes needed to generate reliable evidence, as well as strong and clearly specified scientific methods including proper comparisons and adequate sample sizes. Students wrote and articulated many structurally complex arguments that lacked such markers of argumentation quality, and some structurally limited arguments were rated as high quality on these new dimensions. These findings indicate that our analysis of quality captures dimensions of argumentation not exposed by structural analyses.

Significance: Our presentation will show that epistemic criteria and reliable epistemic processes can be used to evaluate argument quality in ways that go beyond the purely structural analyses of the Toulmin Argument pattern. We discuss how teachers can promote argumentation using our proposed dimensions of argument quality. Finally, we will present scaffolds developed for PRACCIS that promote high quality argumentation along these dimensions.

Authors

• Clark A. Chinn, Rutgers University

• Ravit Golan Duncan, Rutgers University

• Leah C.-C. Hung, Rutgers University

• Ron Rinehart, Rutgers University