Measuring Students' Transfer of Modeling and Meta-Modeling Skills Using an Online Simulation Environment

• In Event: Transfer for Learning in the Classroom: Developing Contexts to Examine STEM Learning Skills and Strategies

Abstract

Computer science and computational thinking are becoming an integral part of the STEM classroom. Students are not only being taught how to use the technology of the 21st century, but how to incorporate skills such as algorithmic thinking, abstraction, and problem decomposition in new contexts and domains. This updated curriculum calls for both innovative instruction and novel tools for assessment. We present a new measure of computational modeling skills and practices, including model interpretation and understanding the practical use and limitations of models, to be used as a complement to an in-classroom modeling curriculum.

The assessment presented here was developed as part of an exploratory study using the Computational, Collaborative STEM (C2STEM) environment, a block-based programming language designed to leverage the synergies of STEM and computer science learning (Hutchins et al., 2018). In this particular implementation, we assessed the transfer of modeling skills of 40 middle-school students who had completed C2STEM units in physics and marine biology. In particular, the assessment was designed to probe students’ ability to transfer their skills to a new environment in a new domain. The assessment had two parts: students were first given time to explore a NetLogo simulation which modeled emergent bird-flocking behavior (Wilensky 1999; Wilensky 1998) and then asked to respond to a set of paper-and-pencil questions. Modifications to the simulation environment and specific meta-modeling constructs were designed in collaboration with the C2STEM team and with the marine-biology classroom teacher, who included framing around how and why scientists use models in practice as part of her classroom instruction.

We find evidence that students were able to identify the functions of the model parameters, despite never working in this particular simulation environment. We also find evidence that students were able to identify directional relationships between model parameters and the emergent behavior. Furthermore, there was evidence that students were able to translate from their learning around block-based code to a text-based environment. Students could identify analogs to the C2STEM “green flag” and “simulation step” as well as answer questions specifically related to lines in the text-based NetLogo code. To investigate student concepts around meta-modeling, students were asked how the flocking simulation may be more realistic. Responses included comments about birds specifically, about the features of the model, and about features external to the model (e.g., weather conditions).

While this exploratory study illuminated interesting student conceptions, it also illustrated important areas of improvement for the assessment design. For example, students seemed unfamiliar with some modeling language. We have since been iterating on how we can change that wording to better target the modeling constructs. In addition, it is possible that students require more scaffolded exploration time to facilitate better transfer of skills. This could include limiting the number of sprites and/or making visual cues so students can better see how changing the model parameters affects the birds’ individual behavior. We continue to iterate on the assessment design and intend to use the assessment in other classroom modeling contexts.

Authors

• Rachel Wolf, Stanford University

• Kristen Pilner Blair, Stanford University

• Doris B. Chin, Stanford University