Mechanism-Focused Approaches to Studying Student Thinking About Social Policy

• In Event: 72.065 - Roundtable Session 30
  In Roundtable Session: 72.065-7 - Computational Methods for Qualitative Learning Sciences Research: Affording Materiality to Text-Based Data

Abstract

Research shows that citizens struggle with identifying policies that they support, and that they often vote for parties that do not support the same policies as they do (Luskin, 1987, 1990). Shtulman and Calabi’s (Shtulman & Calabi, 2008, 2012) work suggests that part of the explanation to this paradox is that citizens fundamentally do not understand the policy issues, and that they therefore are unable to reason about policies to address them. Ranney et al.’s (2012a, 2012b) work on understanding the physics mechanisms behind global warming and attitudes towards policies to address it, suggests that taking a mechanism-based approach to explaining the underlying issues to citizens may help address this gap.

However, social policy issues are often extremely complex systems in which many different mechanisms interact, sometimes reinforcing each other, and other times canceling each other out. Understanding the interactions between these mechanisms in student thinking therefore necessitates not only identifying the individual mechanisms, but also if or how they co-occur, and how students may activate them in conjunction to make sense of policy phenomena.

Data collection and analysis
I conducted this study with undergraduate students in an Introduction to Social Policy-course for Social Policy majors. The main focus of the unit was to forefront human behavior mechanisms that lead to inequity in an urban planning policy design context. A set of five questions was asked two weeks prior and one week after students completed the unit.

The data presented here focuses on one question from the pre- and post-questionnaire: “In 2-4 sentences, how might a person’s income affect their commute time?” This is a complex issue that is impacted by many different, lower-level mechanisms. For instance, income affects whether people own cars, can afford to take more expensive transit options, where they live relative to their workplace, or how well their neighborhood is serviced by transit.

Methods
All pre- and post-responses were manually coded. This first round of coding produced 26 individual mechanisms. Pre- and post-responses were then imported into Python and all responses were clustered using centroid clustering (Sherin, 2013) at the sentence level. Clustered pre- and post-data were then compared for differences in the co-occurrence of codes within clusters.

Results
Preliminary results suggest that student responses centered more on particular codes between pre- and post. In spite of drawing on more mechanisms overall, students drew on fewer mechanisms per sentence. This may indicate that student thinking became more clear and focused on explaining particular sub-phenomena rather than simply listing mechanisms.

Implications and significance
Mechanistic reasoning is gaining increasing attention in the Learning Sciences (e.g. Goldstone & Wilensky, 2008; Russ, Scherr, Hammer, & Mikeska, 2008). This work explores how the computational methods that we use to analyze this particular kind of reasoning must take into account the complex nature of the interactions between these mechanisms, and the ways in which students in their responses articulate these.

Author

• Arthur Hjorth, Northwestern University