Bias and Balance: Examination of Propensity Score Methods With a Larger Treatment Than Comparison Group

• In Event: AERA Virtual Poster Session 7
  In Poster Session: Poster Session

Abstract

Propensity score methods are used to reduce the effect of selection bias on treatment effect estimates. However, recommendations regarding the application of propensity score methods when the treatment group is larger than the comparison group lack empirical support. Using simulated data, we examined the performance of nearest neighbor matching, nearest neighbor matching with 0.20 SD caliper, and generalized boosted modeling, using ATT and ATC coding when the treatment group was larger than the comparison group. Nearest neighbor matching with caliper resulted in adequate group covariate balance and unbiased treatment effect estimates across both coding methods. Generalized boosted modeling and nearest neighbor matching resulted in varied group covariate balance and biased treatment effect estimates, depending upon coding method.

Authors

• Beth Perkins, Association of American Colleges and Universities

• S. Jeanne Horst, CommonLit, Inc.