Investigating the Use of Propensity Score Analysis With Complex Samples

• In Event: Advanced Studies of National and State Databases: Applications and Methodological Considerations

Abstract

The purpose of this study is: a) to examine published educational literature to determine the extent that propensity score analysis has been applied to complex samples and how the sampling design has been addressed in the PSA; and b) to quantify the effects of ignoring the sampling design when using propensity score analysis. Of 163 published studies, 45% had analyzed complex survey data. In review of the 74 studies that had analyzed complex survey data, less than 25% had made some type of adjustment for the sampling design. The results of the PSA demonstration suggest that while the non-overlapping confidence intervals suggested there were no statistically significant differences by CS adjustment, the tests of inference produced important differences to note.

Author

• Debbie L. Hahs-Vaughn, University of Central Florida