AI Predictions Cannot Replace Surveys, but May Augment Them

• In Event: Methodology Section Open Call Paper Session

Abstract

Precision in household surveys is increasingly costly to buy with additional interviews, yet many statistics that inform health, labor, and social policy still require narrow confidence intervals to be useful. Our question is whether we can obtain some of that precision “for free” by exploiting information already available at population scale. Prediction‑Powered Inference (PPI) provides principled framework to achieve this: obtain a predictor using either existing data or external information (LLM), predict it on a very large auxiliary file, and then calibrate the prediction‑based estimate with a gold‑standard correction computed on the survey labels. As a demonstration, we apply PPI to National Health Interview Survey (NHIS) and General Social Survey (GSS), harmonized with the American Community Survey (ACS), covering a broad range of outcomes that include political opinion, socio-economic conditions, and health conditions and behaviours. We show that the combination of auxiliary population-level information and external prediction algorithms can increase the effective sample size of survey data.

Authors

• Lai Wei, The University of Hong Kong

• Yifei Zhu, The University of Hong Kong