Measuring Critical Thinking, Conceptual Understanding, and Cognitive Gain During Augmented Hands-On Informal Science Learning Experiences

• In Event: Assessing Learning in Embodied Mixed-Reality Environments

Abstract

Objectives
Investigators at The Franklin Institute Science Museum and the University of Pennsylvania’s Graduate School of Education are working together to develop protocols for measuring critical thinking skills, conceptual understanding about a science topic, and cognitive gain within the context of an informal science learning experience with hands-on physical devices that have been enhanced with digital augmentations.

Perspectives
There is a need, as documented by the National Research Council report (Bell et al., 2009) as well as others (Rennie, Feher, Dierking, & Falk, 2003), for systematic and rigorous studies of learning designs in informal science education settings in order to understand the real potential for impacting science education. These investigations respond to that call for data in three key areas. First, there is a need for clear evidence that conceptual gains result from informal engagement with science content. Second, research is needed to learn how emerging digital platforms improve the learning experience in informal settings. Finally, data is needed to determine the emergence of higher order cognitive skills such as critical thinking and theorizing through informal science experiences.

Methods
A quasi-experimental design is being used to probe the characteristics and parameters of middle-school (grades 5-8) student interaction with the devices in a variety of imposed learning conditions. The devices (see Figures 1 and 2) offer fixed-location mixed-reality applications that enable direct physical encounters with a scientific phenomenon which is augmented with a digital visualization of the unseen elements of the phenomenon. For example, hands-on play with bar magnets enables the learner to feel the attractive and repulsive forces while a digital visualization simultaneously maps the corresponding electromagnetic fields around the magnets, enabling students to recognize the dynamic nature of the forces. The mixed-reality applications are intended to bridge the divide between the experiential and interpretive aspects of the experience.

Data
Several data sets are being collected. Study participants complete pre-intervention surveys to determine existing conceptual understanding about the science topic at hand. They complete a similar survey post-intervention. During or immediately after their encounter with the mixed reality application, they complete a worksheet that gauges understanding. Observational instruments are also used to capture behavioral characteristics. Finally, interviews are conducted with 20% of the participants. These varied data sources have enabled several different analyses in an attempt to look at changes in critical thinking skills, conceptual understanding, and cognitive gain as a consequence of the intervention. At least four and as many as six different learning conditions have been applied so as to determine the influential factors. Results are forthcoming and will be presented at the symposium

Significance
The evidence is pointing toward the favorability of using mixed-reality applications within the setting of an informal science learning experience. The study’s significance, therefore, is related to the potential transformation of the way that hands-on science learning exhibits are designed and constructed for maximal learning impact. The research framework constructed for this particular investigation may also prove useful in a wider variety of informal learning contexts.

Author

• Karen J. Elinich, The Franklin Institute