Assessing the Complexity in Complex Causal Learning: Discrete Knowledge, Perception, Attention, and Reasoning

• In Event: Developing Evidence of Understanding Complex Systems

Abstract

Students tend to simplify systems and complex causal concepts when reasoning about complexity (Grotzer, 2003; Hmelo-Silver & Pfeffer, 2004; Jacobson, 2001; Penner, 2000; Raia, 2008). This raises critical questions about what we take to be evidence that they are advancing in their understanding and ability to interact with complex systems dynamics.
This poster introduces two methodological challenges involved in assessing student learning of complex systems: 1) figuring out what students know given the human cognitive tendency towards efficiency which results in pruning explanations to their most efficient (and not necessarily complex) forms in communicating them and 2) including perception and attention to complexity in addition to reasoning in assessments of complex systems understanding. Open-ended assessments allow students to structure the concepts but they often fall prey to factors of efficiency confounding students’ ability to reason in particular ways from how far they extend the narrative of their explanation. Formal assessments draw students’ attention to the relationships resulting in cuing that removes perception and attention from the equation. Real world problem-solving requires recognizing opportunities to reason in particular way, yet research shows that such sensitivity is often a significant challenge (Perkins et al, 2000).
This poster illustrates the patterns above with data that was part of a microgenetic study of second, fourth, and sixth graders (n = 12) in a diverse school system in the Boston area serving an ethnically and socio-economically diverse population focused on reasoning about the complex causal pattern of action at a distance. Additional data will be shared from two complementary forms of assessment that may address these challenges: 1) formal paper and pencil assessments focused on discrete features of causal concepts; and 2) informal assessments of student behavior in a virtual world that reveal how students deploy their attention and what they reason about. The formal assessments focused on probabilistic causality and action at a distance and were designed to contrast more and less complex forms of reasoning in ways that did not interact with efficiency of explanation and offered answer choices at scaled levels of complexity that each contained scientifically acceptable explanations. Findings from a study using the assessments with sixth graders from ethnically and socio-economically diverse schools in Lawrence and Cambridge, MA (n = 88), will be shared. Another data set illustrates ways that informal assessments using a multi-user virtual environment (Metcalf, Kamarainen, Tutwiler, Grotzer, & Dede, 2011) can reveal how students perceive and attend to complex phenomena. This analysis focuses on the movements and behaviors of four pairs of middle school students (n = 8) (from a larger study (n = 39) in Cambridge) in the virtual world as they attempted to solve an ecological puzzle. Maps of student movement along with a quantitative accounting of the features and zones where they concentrated their inquiry illustrates changes in how students navigated the space and collected data as they developed more sophisticated notions of the complex systems dynamics as corroborated by their dialogue about their hypotheses and the types of evidence that they sought.

Authors

• Tina A. Grotzer, Harvard University

• Michael Shane Tutwiler, Harvard University

• S. Lynneth Solis, Harvard University

• Kasia Derbiszewska, Harvard University

• Caroline J Courter, University of North Carolina - Wilmington