Asking Questions in the Echo Chamber: How and When Uncertainty Emerges in Education Policy Discourse

• In Event: 56.041 - Theories and Analysis in Relational Governance: Shaping Policy Through Research, Ideas, and Networks

Abstract

Recent studies have demonstrated that the production of policy analysis can be politicized within education as well as other fields (Henig 2009; Rich 2004; Rogers-Dillon 2004). Furthermore, some research has suggested that within education policy debates, an “echo chamber” can develop, “where a relatively small and selective set of studies is repeatedly cited by other researchers and advocacy organizations within an advocacy coalition to advance a policy agenda” (Goldie, Linick, Jabbar, and Lubienski 2014, 282). Goldie et al. (2014) demonstrate the presence of an echo chamber in research on vouchers, while Reckhow and Tompkins-Stange (2015) show the emergence of an echo chamber in research on teacher quality. In research communities, scholars are familiar with the concept of uncertainty—the limits of empirical findings due to statistical error, contextual factors, or unclear implications. Do echo chambers tend to limit attention to uncertainty? When have issues of uncertainty entered the policy debate on teacher quality in the last 15 years? When key actors convey uncertainty, does it impact the views of other actors?
In this study, I analyze congressional hearing testimony on teacher quality from 2000 to 2015 to examine how and when actors express uncertainty and to assess whether uncertainty spreads. Additionally, I draw upon coding of roughly 40 policy reports that were cited by more than one witness in this set of congressional hearings. Uncertainty in both the hearing testimony and reports was coded based on statements that indicated mixed views, a lack of information or adequate research on the issue, unclear implications of research findings, or questions about consequences of implementing the policy.
Overall, the findings point to an overwhelming lack of public discussion about uncertainty related to teacher quality policies. Based on analysis of testimony given by nearly 900 witnesses, expressions of uncertainty are incredibly rare—occurring less than 40 times. Researchers and academics are more likely to express uncertainty than other types of actors (e.g. representatives of advocacy groups, school districts, politicians, etc.) To study the impact of uncertainty (and lack thereof) on the broader debate, I use social network analysis. I examine whether actors who agree on specific policies with an actor who expressed uncertainty are more likely to share uncertainty in future testimony. I find that uncertainty fails to spread—questions that enter the echo chamber are like the tree falling in the forest that no one hears. To the contrary, and not surprisingly, certainty drives the debate and coalition formation. Actors are more likely to adopt preferences expressed by the witnesses who cite reports that lack discussion of uncertainty (based on the coding of reports). Lastly, I find that that expressions of uncertainty vary over time. From roughly 2007 to 2011, certainty was particularly dominant in the debate, and a coalition of actors agreed on many policy ideas related to teacher evaluation reform. Uncertainty was more prevalent prior to 2007, before this coalition came together, and after 2012, when new questions were raised about the implementation of teacher evaluation policies.

Authors

• Sarah Reckhow, Michigan State University

• Sarah Hilary Galey, Michigan State University