The Implications of Covariate Distributional Differences on Generalizability Statistics (Poster 1)

• In Event: Innovations in Statistical Power, Intervention Planning and Design, and Causal Effects

Abstract

Understanding the extent to which a study’s results apply or generalize to a larger population continues to be at the forefront of evaluation research. In practice, researchers have used propensity score methods to assess the generalizability or similarity between a study sample and population, with the understanding that a study’s results will generalize when the two groups are similar. Propensity scores are estimated using observable characteristics or covariates, which can vary in distribution between the sample and population. The current study explores the effect of distributional differences on statistics used to assess generalization. We find that the weakest assessments of generalization occur when covariates are normally distributed in one group but have a skewed distribution in the other.

Authors

• Wendy Chan, University of Pennsylvania

• Jimin Oh, University of Pennsylvania

• Katherine Wilson, University of Pennsylvania