Exploring the Assistance Dilemma in a Simulated Inquiry Learning Environment for Evolution Theory

• In Event: Leveraging Simulation-Based Learning Environments to Support Science Practices

Abstract

Voyage to Galapagos is investigating the question: How much assistance is the right amount to provide to students as they learn with educational technology? To investigate this, we have developed a web-based computer program, Voyage to Galapagos (VTG), to provide middle school students with the opportunity to do simulated science field work in Galapagos while investigating the key principles of evolution.
A key issue in the learning sciences and educational technology is the assistance dilemma: how to find the right balance between, on the one hand, full support and, on the other hand, allowing students to make their own decisions and, at times, mistakes (Koedinger & Aleven, 2007). Clearly there are benefits and costs associated with both ends of this spectrum. The software encourages the student to follow the steps of good scientific inquiry, e.g., developing hypotheses, analyzing data, drawing conclusions, and reveals the basic principles of evolution to the students. The open learning environment provides latitude for variability of student actions – and student errors – allows for a wide variety of assistance, and the ability to either intervene after those actions are taken with help – or not. We are looking at two variables of assistance (frequency of intervention and level of support) and five conditions of assistance that include error flagging, text feedback on errors, hints, and preemptive hints. This provides the foundation for our experimental design.
Learning goals and tasks aligned with the NGSS have been used to create Bayesian Networks to collect data about student actions, assign probabilities of students having made certain errors, and make decisions about error feedback and hints to provide students. Thus, we have been able to create the conditions of assistance that are the focus of our experimental design. The study compares conditions and determines which level of assistance leads to the best learning outcomes, both overall and per different levels of prior knowledge.
Following IRB-approved protocols, Cognitive Labs (12 students) were used to identify usability issues and establish initial construct validity. We conducted Classroom Feasibility Testing with 7th grade classes (161 students) in two schools. Data were collected in the learning management system (LMS) while they worked through the application.
Importantly, the interactions and communication between the flash-based application, the LMS database, and the Bayesian network has been demonstrated to operate effectively in providing real-time feedback and assistance to students. Preliminary analyses of coded data from the LMS, Bayesian network, cognitive labs, and classroom observations revealed that the different experimental conditions can be distinguished and that students with greater assistance advanced further through VTG than those with less assistance. The results are being used iteratively to adjust the sensitivity of the Bayesian network and to inform revisions to the application prior to the pilot study.
This study will help us to better understand how much guidance students need as they learn – and how to cater guidance to the prior knowledge level of students – and thus to be able to appropriately design software to best support student learning.

Authors

• Daniel Brenner, WestEd

• Michael Timms, Australian Council for Educational Research

• Bruce McLaren, Carnegie Mellon University

• Andrew Grillo-Hill, WestEd

• Kim Luttgen, WestEd