Reasoning Based on Evidence Type: An Intervention to Improve Students' Evidence-Based Reasoning

• In Event: Focusing the Analytic Lens on Evidence-Based Reasoning: Where We Are and Where We Need to Go

Abstract

Objectives and Theoretical Frame

Students commonly encounter different types of evidence (e.g., correlational, causal), when reading. Being able to draw appropriate conclusions from evidence of different types is an essential skill for both students’ academic achievement and personal decision making. Despite its importance, such evidence-based reasoning skills have been found to be limited among undergraduate students (e.g., Bleske-Rechek et al., 2015; Woolley et al., 2018). We present the results of an intervention aiming to improve students’ reasoning about different types of evidence. In particular, we sought to foster students’ reasoning about three commonly occurring evidence types (Hornikx, 2005): (a) comparative (i.e., non-experimentally comparing two groups), (b) correlational (i.e., describing an associative relationship between two continuous variables), and (c) causal (i.e., describing a cause-and-effect relationship between two variables, based on random assignment). Through the Evidence-Based Reasoning (EBR) intervention we sought to determine if we could improve students’ identification of these three evidence types and students’ abilities to draw appropriate evidence-based conclusions.

Methods

Participants were 107 undergraduate students enrolled in a large university in the Northeastern United States (age: M=19.00, SD=0.97). Participants competed the EBR intervention as a course assignment, and were provided with the option to assent to sharing their data with researchers. Students first completed a pre-test assessing their knowledge about various types of evidence. Then all students completed the EBR intervention. Following completion of all intervention activities, students completed a post-test (in parallel form to the pre-test).
The EBR intervention was divided into four modules, introducing students to (a) comparative evidence, (b) correlational evidence, (c) causal evidence, and (d) how to draw evidence-based conclusions. Within each module, explicit definitions and examples were presented to teach students about each target evidence type. Practice tasks of different forms (e.g., multiple-choice, open-ended questions) were also included within each module. Tailored feedback was provided for multiple-choice practice questions and model responses were presented in association with open-ended questions.

Results

A paired-sample t-test was used to examine differences in performance on the pre-test and post-test measures. Students performed significantly better on the post-test measure (M=0.88, SE= 0.01) than on the pre-test measure (M=0.57, SE=0.01), t(106)=21.21, p<.001, Cohen’s d = 2.05. We further examined students’ sub-scores on questions asking them (a) about research design (e.g., dependent and independent variables) and (b) to identify different types of evidence, and (c) to draw appropriate evidence-based conclusions, assessed at pre-test and post-test. Significant differences were found on all three question subsets, ps<.001. Please see Table 2 for detailed results.

Significance

We developed an EBR intervention that successfully improved students’ abilities to identify different types of evidence and to draw accurate evidence-based conclusions. Iterative steps in intervention design and directions for future implementation and research will be discussed.

Authors

• Hongcui Du, Florida State University

• Alexandra List, Pennsylvania State University