Interviews, Diagrams, and Movement Analysis: Assessing Learning in a Mixed-Reality Simulation of Planetary Astronomy

• In Event: Assessing Learning in Embodied Mixed-Reality Environments

Abstract

In this presentation we will describe the approaches taken to assess learning in a study of a mixed-reality science center experience designed to build middle school students’ intuitive understanding of physics concepts. We have designed and implemented a simulation of planetary astronomy where children “become” part of the system and act out their predictions of how elements (e.g., an asteroid) will behave (Figure 1). Floor-projected imagery and motion tracking facilitate an immersive simulation experience that builds off the known affordances of body movement for thinking and learning (Gallagher, 2005).

A number of recent reports have highlighted the benefits of informal and simulation-based learning experience for science education (Bell et al., 2009; Honey & Hilton, 2011), but they have also stressed the need for novel and comprehensive assessment approaches in order detect the kinds of knowledge and skills that are generated by these experiences. In examining the effects of using our MR simulation we have adopted an approach consistent with a mixed-methods study design (Creswell, 2009) and applied an array of both qualitative and quantitative measures to achieve a comprehensive picture of the learning impact. The three measures that will be described in this presentation are structured interviews, a diagramming activity, and an analysis of participant movement within the simulation.

1. Interviews. Participants in our study are interviewed both before and after their use of the simulation to assess their understanding of pertinent physics principles. We are asking loosely structured questions (e.g., What happens to a ball thrown in space?) as well as questions about phenomena observed in the simulation (e.g., How would you describe the shape of an object’s orbit around a planet?). Answers to these questions, including the gestures they use when articulating their thinking, are helping us to understand the impact of using the embodied simulation on their ability to reason about complex science concepts.

2. Diagrams. Participants are being asked to draw sketches of certain phases of the simulation with relatively little specification of how or what they should draw. These diagrams provide insight into what the participant felt was important and their overall level of comprehension.

3. Movement. Performance in our simulation is determined by how participants move, and thus we are able to infer the efficiency of a child’s learning by examining how quickly they adopt more expert-like behavior. Do they, for example, perseverate on their initial intuitions? Do they make incremental adjustments?

We are currently collecting data from several hundred participants, but a pilot study of 62 middle school students revealed some interesting trends concerning data using the methods described above. For example, children who used the MR simulation tended to include more dynamic elements (e.g., arrows) and less “surface features” in their diagrams than participants who used a desktop computer version of the same simulation, F = 5.78, p = .019. We believe that the use of multiple approaches to assessment will ultimately result in a more complete and nuanced understanding of how science learning occurs in mixed reality environments.

Authors

• Robb Lindgren, University of Central Florida

• J. Michael Moshell, University of Central Florida