Loosely Coordinated Studies in Evidence-Based Policymaking

• In Event: The Problem of Knowledge Cumulation

Abstract

The Foundations for Evidence-based Policymaking Act of 2019 in the US Government has accelerated the number policy evaluations occurring in the federal government. Even before the act passed, the Office of Evaluation Sciences had executed nearly 70 randomized controlled trials using administrative data to measure policy-relevant behavior change and implementing new designs for state-citizen relations inspired by the behavioral science literature and built on the experience of civil servants. This evidence-based policy movement is only increasing in scope and reach across levels of government and the world. At this point hundreds of RCTs have been run around the world aiming to inform policy-makers about their next steps in decision making: vaccination communication is one such area where many studies have been done, but where the studies have not been directly coordinated (say, following the EGAP Metaketa model), but where there scientific cumulation beckons. Taking the example of attempts to encourage vaccination across institutions and the example of a loosely coordinated study of early childhood language learning (the Providence Talks Replication Study), we offer some reflections on how the scientific workflow might need to change in order for future collections of studies to be most policy relevant while also most effectively and quickly advancing scientific explanation. For example, we show that transparency and replicability principles arising to serve the sciences are not directly political enough for use in policy and we advocate tighter coordination between ethnographic teams of human centered designers, social scientists, and data scientists.

Author

• Jacob Bowers, University of Illinois, Urbana-Champaign