From Noisy Data to Reliable County Disability Estimates: Improving Precision with Bayesian Methods

• In Event: Measuring What Matters: Improving Data Quality and Decision-Making under Uncertainty

Abstract

County-level estimates of disability prevalence and employment rates by disability status are essential for informing policy decisions, targeting services, and allocating resources across communities. The American Community Survey (ACS) is a primary source for such local estimates, but even five-year aggregated data can be highly unreliable for smaller counties and demographic subgroups due to small sample sizes and large margins of error. These limitations can obscure meaningful geographic and demographic patterns and hinder effective policy and program design.

This study addresses the challenge of producing reliable county-level estimates of disability prevalence and employment rates by demographic group. We ask how researchers can improve the stability, precision, and reliability of ACS-based estimates, particularly for small population groups, while relying only on publicly available data.

We apply Bayesian small-area estimation techniques to ACS five-year summary tables from IPUMS NHGIS, focusing on three key measures: prevalence of any disability, prevalence of self-care disability, and employment rates by disability status. Our approach incorporates margins of error from published ACS estimates to explicitly account for uncertainty and uses hierarchical modeling to borrow strength across counties and demographic groups. This allows estimates from data-sparse cells to be informed by broader patterns while preserving meaningful variation, thereby improving the usability of these estimates for decision-making in data-sparse contexts. We assess improvements by comparing Bayesian estimates to raw (frequentist) ACS estimates using metrics such as ranges, margins of error, and coefficients of variation.

The Bayesian estimates are consistently more stable, precise, and reliable than raw ACS estimates. The approach reduces extreme and implausible values and lowers margins of error across the vast majority of county-demographic cells, with particularly large improvements for small populations such as young children. The proportion of unreliable estimates declines substantially for disability prevalence measures. These gains reveal clearer geographic and demographic patterns, such as age gradients in disability prevalence and regional variation, that are often obscured in raw data and are directly relevant for identifying areas of need and targeting interventions.

By improving the quality of widely used ACS-based estimates, this approach provides a more reliable foundation for policy analysis, service planning, and resource allocation, especially in smaller or rural counties where data limitations are most severe. Because the method relies solely on publicly available summary tables and scalable Bayesian techniques, it is easily reproducible and extendable to other policy-relevant indicators, such as Medicaid or SNAP participation. These advances enable more accurate monitoring of disparities and support better-informed, data-driven decision-making under uncertainty.

Author

• Yonatan Ben-Shalom, Mathematica
  Presenting Author