What Kind of Science Public Managers Actually Use to Make Decisions, and How

• In Event: Politics of Science and Technology Here and Elsewhere

Abstract

Scientific data and analysis inform myriad regulatory decisions such as allowable water pollution levels, drinking water standards, and food handling and processing rules. Moreover, agencies rely on a vast array of scientific inputs in assessments used to design infrastructure, make plans, and support policy decisions in legal conflicts. Despite having vast informational needs, there is limited systematic understanding of science consumption by agencies, particularly field offices tasked with implementing agency policy. The quality of the academic literature used in decision-making, the impact of open-access sources, and the relative prominence of research produced in-house – among other characteristics of agencies’ science consumption – are largely unknown. A major barrier to developing knowledge on the use of science in government decision-making is the academic community’s preoccupation with self-assessment and knowledge diffusion within the academy. That is, studies of scientific impact and popular measures of scholarly contribution such as Hirsch’s h-index rely on bibliometric analysis tools that extract citations and analyze co-reference patterns from peer-reviewed academic publications only. We turn away from navel gazing to consider knowledge diffusion within public agencies by (a) developing a bibliometric tool capable of extracting citations of peer-reviewed and other types of publications and (b) creating a measure that analyzes citations’ policy and management impact (“m-index”), or common usage among public managers. We apply our tool and measure to thousands of environmental assessments produced by the US Forest Service to describe and explain why the quantity, novelty, quality, and impact (as defined by either the h- and m-index) of science used varies across actions and field office settings. In addition to advancing ways to evaluate how science informs government decision-making, our work reveals the extent to which impact in the academy is or is not equivalent to impact in practice.

Authors

• Tyler Andrew Scott, University of California, Davis

• Gwen Arnold, University of California, Davis

• Forrest Fleischman, University of Minnesota

• Cory Struthers, University of Georgia