Examining Variation in Achievement Impacts Across California’s Full-Time Virtual Schools

• In Event: Virtual Schools in the United States 2014: Politics, Performance, Policy, and Research Evidence

Abstract

Perhaps the most significant current trend in education reform is the growth of virtual (online) schools (Watson et al., 2011, 2012). While these schools offer the potential to radically restructure the way that teaching and learning happens, they also present challenges for researchers and policymakers who want to know whether they work. Specifically, the extent to which virtual schools depart from traditional brick-and-mortar schools creates difficulties with respect to assessing what these schools are doing in terms of teaching and learning and how well they are doing it.
This study uses longitudinal student-level data covering all full-time virtual schools (thirty-two total) in California from 2010-2012 to study the effect of virtual schools on student performance. Based on our web-based research, all full-time virtual schools in California are contracted to run as charter schools. Full-time virtual schools are defined as those schools in which instruction is delivered entirely or primarily through online methods (Watson et al., 2012). However, students self-select into virtual schools, making it difficult to estimate the effects of these schools on achievement. This study addresses this challenge using propensity score matching (PSM). Following the counterfactual framework (Rosenbaum & Rubin 1983; Rubin 1974), PSM matches virtual school students (“treatment” group) to those who are non-virtual school students but similar in all other preexisting observed characteristics (“control” group), based on a propensity to attend a full-time virtual school. In addition, this study addresses selection bias due to both observed and unobserved covariates. Previous studies employing the PSM approach have focused mainly on selection bias due to observed covariates (Chevalier & Viitanen, 2003). Using the Rosenbaum bounds method (Rosenbaum 2002), we evaluate the extent to which selection bias on unobserved covariates would nullify propensity score matching estimates of the effects of virtual schools.
The data for this study come from the California Department of Education (CDOE). The Department maintains longitudinal records on all public school teachers and students, including test scores (CALPADS), demographic data, enrollment and attendance information. This study supports our larger project’s focus on heterogeneity in the achievement effects of charter school attendance across demographic groups in California. We focus on the difference in impact between virtual school students and non-virtual school students in our sample, primarily because of the finding of large positive impacts in urban charters and non-significant or negative impacts in non-urban charters has been noted in our prior analysis.
Although virtual schooling is gaining ground in the K-12 classroom (Molnar et al., 2013), its impact on academic performance remains largely unexplored. Considered one of the largest markets of virtual school programs in the United States, California offers a fertile context for the study of virtual school impacts and thus serves as the focus of our study. Ultimately, our study focuses on discovering which policy-amenable aspects of virtual schools—their characteristics and conditions— are related to their ability to maximize student learning and close the achievement gap.

Authors

• Charisse Atibagos Gulosino, University of Memphis

• Jonah Liebert, Teachers College, Columbia University