Propensity Score Analysis When Treatment Group Is Larger Than Comparison Group: An Applied Assessment Example

• In Event: Statistical Methods and Quantitative Analysis in Higher Education Assessment

Abstract

Propensity score matching (PSM) is useful in educational research when random assignment is not feasible. Ideally, for PSM the comparison group is larger than the treatment group. However, researchers may encounter the reverse. We explored six PSM methods, under the condition of larger treatment than comparison group sizes. Performance on an institution-wide accountability test was compared for students attending a make-up assessment-testing session (N=342; treatment) with those attending their assigned session (N=207; comparison). Propensity for treatment was estimated from demographic, ability, and course experience covariates. Generalized boosted modeling (GBM) resulted in best group balance with no loss of sample size. Groups did not significantly or practically differ on knowledge test scores after GBM, strengthening inferences faculty could draw about learning.

Authors

• Beth Perkins, Association of American Colleges and Universities

• S. Jeanne Horst, CommonLit, Inc.