Using Simulations to Foster Integrated Understanding of Complex, Dynamic, Interactive Human Body Systems

• In Event: Leveraging Simulation-Based Learning Environments to Support Science Practices

Abstract

The goal of the Human Body Systems (HBS) project is to enable high school biology students to develop multilevel mental models of the complex systems of the human body. As students use three model-based instructional modules, they reason about tasks, generate and test predictions, and use evidence to revise their mental models, as scientists do (Nersessian, 2008). This paper focuses on the design and testing of the summative benchmark assessment we will use to assess their ability to do so.

We use evidence-centered design (Behren et al, 2004) to ensure coherence across instruction and assessment and alignment to the NGSS. Model-based learning (Buckley, 2012) and systems thinking (Goldstone, 2006) guide design of student and task models. We specify the student model in terms of system components that interact to produce emergent behaviors at scales ranging from molecules to the whole body. We specify the task model in storyboards of tasks that engage students in model-based reasoning as they use, test, and revise their mental models to interact with multilevel, linked representations. The evidence model specifies data collected and scored as students accomplish tasks.

Evidence-centered design methodology guides domain and curriculum analyses and alignment to NGSS, providing a foundation for the Target Model, which specifies the system levels, core ideas, science practices, and cross-cutting elements to be integrated in the instructional modules and assessed in the benchmark. Creating increasingly detailed specifications, we designed the HBS benchmark assessment as a progression of tasks to elicit students’ understanding as they investigate the extraction of molecules from food, their use in cell processes, and the impact on body functioning.

Data were collected via external reviews for accuracy and appropriateness by our advisors, AAAS, and teachers; think-alouds (eight students) for usability and initial construct validity, and classroom testing for feasibility and psychometric quality. Using IRB-approved protocols, we observed classes and the learning management system (LMS) collected data while 60 students worked through the benchmark. LMS data included students’ answers and their manipulation of models. IRT analysis of the LMS data examined item performance.

Results of curriculum analyses indicated that typical textbooks and curricula cover HBS targets at different times; cells may be covered in autumn with human body systems not covered until spring. Therefore, HBS is designed for use at the end of a student’s first biology course or at the beginning of an advanced course. Think-aloud data analysis indicated few usability issues and well-aligned items. Classroom testing demonstrated the feasibility of delivering HBS to classrooms via the Internet. Students were actively engaged with the tasks, which was confirmed by LMS data analysis.
IRT analyses of LMS data indicated that while all items had acceptable fit, many had poor discrimination. Wright maps revealed that a large group of items were very easy for these students. Triangulation across the data have identified specific issues to be resolved prior to pilot testing.

The close alignment of HBS project goals to NGSS performance expectations make this project an important contributor to efforts that improve science learning.

Authors

• Barbara C. Buckley, WestEd

• Michelle LaMar, University of California - Berkeley

• George E. DeBoer, American Association for the Advancement of Science