Assessing Trade-Offs Between Observational and Experimental Designs for Charter School Research

• In Event: New Directions in Charter School Practice and Evaluation

Abstract

Considerable attention has been paid to observational studies by the Stanford Center for Research on Education Outcomes (CREDO), which have analyzed charter performance in 27 states. However, the sensitivity of CREDO’s findings to its methodology has not been subject to a thorough, independent evaluation. Our analysis reveals four main findings that increase confidence in CREDO’s results. 1) Its use of multiple observations when matching students does not materially affect results; 2) Student participation in programs such as free/reduced price lunch is inconsistently measured across sectors, but matching on these variables only modestly affects CREDO’s estimates. 3) Exogenous instrumental variable estimates of charter effectiveness produce qualitatively similar results to CREDO’s estimates. 4) Impact estimates differ for oversubscribed and undersubscribed charters.

Authors

• Anna J. Egalite, North Carolina State University

• Matthew Ackerman, Harvard University