SimScientists: Interactive Simulation-Based Science Learning Environments

• In Event: Leveraging Simulation-Based Learning Environments to Support Science Practices

Abstract

Objectives
This presentation will describe the theory and research underpinning development of SimScientists simulation-based instructional supplements, the iterative development process, and findings from pilot testing in middle school science classrooms.

Theoretical Framework
SimScientists modules integrate model-based learning and evidence-centered design. The simulations are based on a cross-cutting model of all science systems-- components, interactions, and emergent system behavior. Evidence-centered design was employed to specify the student models (targeted knowledge and skills), evidence models (how evidence of learning will be scored and reported), and task models (features of activities that elicit evidence of learning).
Simulation-based modules were developed for middle school science units on ecosystems and atoms and molecules. The modules contained: (1) a simulation environment modeling a science system, (2) problem-driven inquiry, (3) embedded formative assessments with an intelligent scoring system that provided immediate feedback and coaching, (4) offline self-assessment and reflection activities to promote transfer, collaboration and communication, (5) end-of-unit benchmark assessments, (7) pre/post tests, and (8) professional development.

Methods
The SimScientists instructional modules proceeded through a systematic design and development process. National science frameworks and standards shaped the science knowledge and practices in the student models of what would be taught and assessed. Following review by AAAS, storyboards were developed. Cognitive labs with students and teachers were conducted with the programmed modules. Two cycles of pilot testing were conducted in California and Nevada middle school classrooms.

Data
Formative evaluation addressed the modules’ quality, feasibility of implementation, and impacts on learning. In Pilot Test 2, two teachers and 237 students used the Ecosystems modules, four teachers and 331 students tried the Atoms and Molecules modules.
Data were collected from pre/posttests and benchmark assessments. Case studies included observations, cognitive labs, and teacher surveys and interviews.

Results
Ecosystems. Students posted significant gains from pre to posttest as determined by repeated measures analyses (p<.01). The reliability of all three summative measures was very good. The instructional modules were significant predictors of performance on both the benchmark and the posttest, holding pre test constant.
Atoms and Molecules. Students demonstrated significant gains from pre to posttest as determined by repeated measures analyses (p<.01). The Cronbach’s alpha reliability of the summative measures was very good, but the Rasch reliability of the Posttest was marginal, perhaps due to ceiling effects. Pretest performance, ethnicity, IEP status, FRL status, and class ID were significant predictors of performance on the benchmark. Performance on the instructional modules was more strongly correlated with performance on the benchmark than on the posttest.

Scientific or scholarly significance
The SimScientists project produced strong findings of the quality, feasibility, and instructional impacts of the simulation-based instructional modules. Students learned and were engaged. Teachers were positive about the value of the modules. The SimScientists project was designed to break the mold of the prevailing model of static, text-based transmission of science knowledge by creating a powerful, dynamic intervention that addresses the urgent need to promote deep science learning.

Authors

• Matt Silberglitt, WestEd

• Mark Loveland, WestEd

• Edys S. Quellmalz, WestEd