Time after time: Lessons from three behavioral science research portfolios in government

Session Submission Type: Panel

Abstract

Applications of behavioral science in government can often benefit from building evidence over time and iteratively learning to deliver results. The Office of Evaluation Sciences (OES), part of the U.S. General Services Administration (GSA), is uniquely positioned to build a portfolio of evidence using behavioral science to support government priorities. This panel discussion will describe how OES builds evidence with federal agency partners by incorporating iterative applications of behavioral science and research methods, learning from descriptive studies to inform intervention design, and linking behavioral observations from administrative data to downstream outcomes.

Each of the four panelists will describe a portfolio of evaluations that demonstrate how OES scales evidence building in government: 

Measuring the health effects of Supplemental Security Income (SSI): A 2017 randomized evaluation with the Social Security Administration resulted in an increase in SSI enrollment of 1.8 percentage points (https://oes.gsa.gov/assets/publications/1723.pdf). A follow-up evaluation demonstrated that enrollment in SSI due to the letter reduced the likelihood of death during the evaluation period. The ability to learn whether SSI enrollment affects downstream mortality outcomes was possible because of the availability of high-quality administrative data spanning multiple years and dialogue with academic researchers to apply advanced methodologies. https://oes.gsa.gov/projects/ssi-health-effects/

Supporting small businesses in the federal marketplace: OES is partnering with the Small Business Administration (SBA) to help small businesses compete for federal contracts. Two descriptive studies revealed that small manufacturing and rural businesses are underregistered in SAM.gov (a requirement for securing federal contracts) and are not participating in SBA certification programs. In parallel with this descriptive work, a test of automated market research (https://oes.gsa.gov/projects/2405-automating-market-research/) for federal contracting officers did not significantly increase contracts to small businesses. And, a test of templated capabilities statements (https://oes.gsa.gov/2503-increasing-small-manufacturer-competitiveness/) did not increase federal buyers’ engagement with small businesses. OES and SBA are building on these findings to learn which interventions provide the greatest return on investment in terms of helping small businesses win more contracts and expand their capacity.

Reducing tax filing errors: OES and the Internal Revenue Service (IRS) collaborated to send different versions of informational letters to paid tax preparers and to clients of tax preparers who likely previously made errors on client returns. Sending these letters resulted in an estimated $144 million in cost savings. Follow-up evaluations of letters to preparers and clients can build on the results of these evaluations because of the repeated nature of tax filing and administrative data that can be linked across years and between tax filers and preparers. https://oes.gsa.gov/-reducing-filing-errors-via-outreach-to-preparers/ 

Policy Nudges Nationwide: In this paper, we report the results from three large-scale field experiments, conducted in partnership with the U.S. Department of Treasury, which involved varying the email messaging used to request timely reporting by recipients of American Rescue Plan (ARP) funding. The subjects of the experiments were the full population of over 20,000 small towns and cities receiving ARP funding.

Policy Area

• Methods and Tools of Analysis

Chair

• Susan Jenkins, Evidence Oracle, LLC

Discussants

• Susan Jenkins, Evidence Oracle, LLC

• Elizabeth Linos, Harvard University

Organizer

• Elana Safran, Office of Evaluation Sciences / U.S. General Services Administration

Individual Presentations

• Measuring the health effects of Supplemental Security Income (SSI) - Presenting Author: Elana Safran, Office of Evaluation Sciences / U.S. General Services Administration

• Supporting small businesses in the federal marketplace - Presenting Author: Shibeal Leigh O' Flaherty, Office of Evaluation Sciences

• Reducing tax filing errors - Presenting Author: Julia K Brown, Office of Evaluation Sciences, U.S. General Services Administration

• Policy Nudges Nationwide: Using Behavioral Interventions to Improve Timely Reporting in Government - Presenting Author: Lizzie Martin, University of Pennsylvania