Data Talks but Models Lie? Studying Stakeholders’ Epistemological Stances Toward Digital Media in Planning

• In Event: Distributing, Learning, and Making STEM Arguments in Public Media

Abstract

The purpose of this research project was to better understand how interactive simulations can be used directly by stakeholders in groundwater planning meetings. Collaborative planning models differ from top-down “rational planning” approaches in their commitment to active public participation. As the range of digital tools available to planners has grown, however, so has the challenge of incorporating stakeholders into planning processes. Simulations like Agent-Based Models (ABM) can help planners explore creative plans and gain important insights by representing the interactions present in complex human-natural systems (Zellner, 2007, 2008). These benefits accrue primarily to the users of the tool, though, implying that stakeholders need to be directly involved in the process of using these digital interactives (Becu et al., 2007). To that end, we conducted a series of workshops near a large Midwestern city, where stakeholders co-constructed an Agent-Based Model (ABM) with our research team to explore how their suburban communities might deal with dwindling groundwater supplies.

We examined learning processes exhibited by stakeholders along four dimensions relevant to the special characteristics of ABM tools: understanding complexity, understanding modeling, shifting from individual to shared perspectives, innovating policy(Zellner et al., 2012). For analysis, dialogue was transcribed, segmented into turns, and time-coded. Transcript analyses tracked the ways each ABM model mediated ongoing planning discussions, and modes of interaction among participants as meetings progressed. Two sets of codes were developed to track these foci, one focused on discursive targets (conceptual categories of speech referents), and the other focused on discursive moves (functions of turns in positioning speakers vis a vis emergent observations, explanations or proposals). Each transcript was coded twice independently and Cohen’s Kappa used to determine inter-coder reliability. Pre- and post-test phone interviews established a baseline relative to the four learning dimensions, and assessed changes with respect to these dimensions.

Our findings show that the stakeholders successfully used the ABM to explore community issues, and even reached some important insights about population growth and the need to involve neighboring communities in groundwater planning. Nonetheless, their dialogue also revealed interesting epistemological attitudes towards models and data: (1) stakeholders seemed to privilege "real data" over models of systems, even when the "real data” were incomplete, had provenance issues, or may not have been relevant; and (2) when stakeholders were confronted with a model prediction they disagree with, they tended to discredit the model or put off grappling with that prediction in favor of getting "more data." This reliance on data to provide answers is troubling - arguably, models are what are ultimately needed to help guide decision-making, as they contain causal connections while data alone does not (Innes, 1990; Wong, 2005). In an era of “big data” and interactive visualizations there may be an unfortunate tendency to de-emphasize the importance of models. And yet models implicitly underlie any reasoning we might try to do with data visualizations, whether or not they get acknowledged or represented. Our results suggest that data literacy and model literacy are interacting spheres of learning relevant to participatory planning.

Authors

• Leilah Lyons, University of Illinois at Chicago

• Joshua L. Radinsky, University of Illinois at Chicago

• Moira Zellner, University of Illinois at Chicago

• Dan Milz, University of Illinois at Chicago