Where Does the Evidence Come From? Mapping the Geographic Footprint of Evidence Federal Clearinghouses

• In Event: Spatial Methods and Geographic Dimensions of Policy Evidence

Abstract

The potential for geographic analysis to improve policy research extends well beyond mapping. In policy studies that rely on quasi-experimental designs geography is used to create the variation needed to isolate causal effect from confounding factors. In experimental designs geography often structures treatment assignment through cluster randomization, defines the boundaries within which spillovers threaten key causal assumptions, and impacts the scalability of findings across locations. More fundamentally, evidence about the efficacy of policy interventions is always collected from somewhere and is therefore structured by context-specific factors.

Federal evidence clearinghouses, such as the What Works Clearinghouse and Teen Pregnancy Prevention Evidence Review, have become central infrastructure for translating policy research into practice and transferring policies across locations. These repositories catalog studies, rate the strength of their causal evidence, and signal to state and local decision makers which interventions merit adoption. Yet these clearinghouses treat geography as incidental, indexing studies by intervention type, population, and methodological rigor while giving little attention to the places where evidence was generated or the features of those places that may have shaped results. Collecting evidence from many kinds of places could in theory reveal how interventions work across contexts, but only if contextual characteristics are tracked and sites are intentionally diversified. In practice, funding structures, evaluation constraints, and limited visibility into existing coverage leave the evidence base geographically uneven and poorly matched to the range of state contexts in which policies are adopted.

In this study, we examine the geographic coverage of evidence currently stored in federal evidence clearinghouses. We begin by constructing the spatial data absent from many clearinghouses, identifying where policy evaluations took place and linking each study to characteristics of the places where it was conducted. Using this newly assembled dataset, we document the geographic footprint of the evidence base across several major clearinghouses, describing where evidence is concentrated or sparse and how study sites compare to the states and populations clearinghouse recommendations target. We then assess how well the contextual features of evidence-generating places align with those of states currently weighing adoption of the same interventions.

Where Does the Evidence Come From? Mapping the Geographic Footprint of Federal Policy ClearinghousesOur findings speak directly to the responsibilities of policy scholars in an era of devolved federalism. As states take on a larger share of policymaking authority and turn to clearinghouses and peer states for guidance, the geographic unevenness of the underlying evidence base becomes a substantive problem for effective policy transfer. We conclude by proposing concrete changes to how clearinghouses could catalog, display, and prioritize geographic information, and by outlining a research agenda in which the intentional geographic diversification of policy evaluation is treated as core infrastructure for evidence-based state policymaking.

Author

• Peter Kedron, University of California, Santa Barbara
  Presenting Author