Rescuing "Bad Data": How Students Use Simulations to Reason Adaptively With Poor-Quality Experimental Data

• In Event: Thinking Across Representations: Designing to Support Investigations That Leverage Physical and Computational Modes of Inquiry

Abstract

Objectives. In learning environments where students design their own experiments, the quality of data they obtain can be poor for a variety of reasons (low replicates, faulty design, experimental error). Such poor-quality data can limit opportunities to reason, allowing students to concede that their results are “inconclusive.” We examine how undergraduate students use simulations to continue to engage with experimental data despite its poor quality.

Theoretical Framework. Pickering (1995) describes scientific practice as emerging from scientists’ accommodations to resistances – blocks to progress that provoke reflection and action. Manz (2015) has argued that designing resistances into curricula can be a means of provoking productive accommodations from students.
Designs that couple experiments and simulations can create many opportunities to encounter resistances as well as multiple avenues for accommodating to them (Author, 2016; Author, 2018; MacLeod & Nersessian, 2013). Thus, in such learning environments we may see students enacting flexible and adaptive responses to problems they encounter – demonstrating what Ford (2008) described as a grasp of practice.

Methods and Data. The context of this study is in an undergraduate laboratory course in which students explore biological study systems using coupled experiments and simulations. In this study, we examined laboratory reports from a 3-week unit in which students conducted experiments with two strains of bacteria that mutate at different rates and explored a computer simulation in which they could compete virtual strains under different conditions and parameter settings.

Research Question: In what ways do undergraduate students use simulations to accommodate to poor-quality experimental data?

We selected reports from two groups, both of whom reported that their data were of poor quality (Group 1 had data from only one treatment, and Group 2 had no data at all). We first used the lab reports to characterize different accommodations to the failed experiments. We then examined each student’s report individually to identify the presence or absence of each accommodation.

Results and significance. We highlight two results. The first is that students made many productive accommodations to their failed experiments (Table 1). Indeed, some students used their investigations in the simulation to argue that their initial experimental predictions were incorrect and went so far as to fabricate “hypothetical” data to support revised predictions. Other students used the simulation to identify and explore novel questions not originally proposed in the experimental system. The second result is that there was a high degree of variation in how students accommodated to their failed experiments, both between and within groups (Table 2). For example, in Group 1, two students (S1 and S3) discussed how the simulation related to their original experiment. However, S2 chose to spend the majority of her report around a novel question that arose from the simulation.
Overall these two results suggest that the coupling of experiments and simulations supports flexible and adaptive reasoning and decision making even when (or perhaps because) experiments fail and suggests the potential for coupled approaches to support students’ developing grasp of scientific practice.

Authors

• Julia Svoboda Gouvea, Tufts University

• Katherine Anne Parker, Tufts University