Standards for Evaluating Evidence in Arguments

• In Event: Argument Evaluation in Education

Abstract

OBJECTIVES AND THEORETICAL FRAMEWORK. In this paper we discuss standards for evaluating arguments. We focus on a critical aspect of evaluating arguments—evaluating the evidence used in arguments. Although evidence is critical to many types of arguments, researchers have as yet not investigated critical issues such as what counts as evidence, what counts as strong (versus weak) evidence, or the criteria that distinguish strong or weak evidence.

We approach standards for evaluating evidence in arguments from both normative and descriptive perspectives. Normatively, we consider philosophical perspectives on quality of evidence. We draw on philosophical literatures to develop a framework of criteria for distinguishing between strong and weak, or good and bad evidence.

Descriptively, we present data on how seventh graders in a model-based inquiry curriculum use evidence in arguments and how they differentiate stronger from weaker arguments. We show that, collectively, seventh graders deploy many of the same criteria for evaluating evidence that scientists do.

DATA SOURCES AND METHODS. Our descriptive analyses derive from data from two year-long studies involving two cohorts of seventh graders (over 500 students) in 26 science classes taught by seven teachers who were implementing a model-based inquiry curriculum. The curriculum included an array of scaffolds to support students’ oral and written argumentation. Each class was video recorded on a majority of days during the academic year. We have sampled hundreds of arguments from class discussions, group discussions, and written work to investigate students’ standards for evaluating evidence in arguments.

RESULTS. Our normative analysis of philosophical criteria for evidence and argument quality and our descriptive analysis of the criteria used by middle-school students converged on a common cluster of principles for evaluating evidence, including: (1) Methodological strength. Normatively, strong evidence is produced via good methods. Seventh graders were collectively aware of many normative methodological criteria, including sample size, reliable measurement. (2) Diagnosticity. The diagnostic power of evidence to distinguish between alternative models is recognized as relevant in both normative and in the seventh graders’ evaluation of evidence and arguments. (3) Relevance. A number of seventh graders took account of the relevance of the evidence—a criterion traditionally considered as normative in work on argumentation. (4) Core processes. Strong evidence was often seen to touch on core rather than peripheral processes in a scientific model. (5) Contextualization. A number of seventh graders exhibited sensitivity to the contextual variation in how evidential criteria are applied.

SIGNIFICANCE. Given the importance of evidence in reasoning and argumentation, intensive work on how students understand evidence is overdue. Drawing on philosophical work, our first contribution is to identify normative criteria for strong (versus weak) evidence that have been used by scientists and scholars in other fields. Our second contribution is to investigate students’ use of evidence in arguments. Collectively, seventh graders show awareness of many normative criteria in their own evaluations of evidence and arguments. However, the proportion of seventh graders that espouse any one criterion is typically substantially less than a majority; thus, they can profit greatly from instruction.

Author

• Clark A. Chinn, Rutgers University