Lessons from the Design and Implementation of a High-Autonomy Literacy Reform in California

• In Event: Effectiveness of Early Literacy Policies: Evidence from Four States

Abstract

Past federal and state initiatives in early literacy have been oft criticized as expensive, prescriptive, or unequal (French, 2020; Stevens, 2003; Waddell, 2011). Presented with critiques of comprehensive policies, states are increasingly turning to an alternate approach in which states provide money and minimal guidance to local stakeholders who then preserve the authority to select programming with limited oversight. This research illuminates the successes and challenges of such a strategy, offering lessons for other states considering similar policies with similar designs.

The Literacy Coaches and Reading Specialists Grant (LCRSG) allows an examination of state-level literacy policy that provides a high level of autonomy to schools. Specifically, California allocated about $300 per student to 818 high-need schools to select from a wide range of permissible literacy activities from professional development to coaching to tutoring, among others. Schools also maintained autonomy to decide when to use these funds at any time over a five-year period. In addition, school sites received professional development, centrally provided by a state-selected contractor, on the science of reading. 

This mixed methods study answers the following research questions: 

1.     What was the effect of LCRSG on student achievement in ELA and math? 

2.     How did schools spend their LCRSG funds? 

3.     What did schools describe as the primary challenges and successes of LCRSG? 

This paper leverages the program feature in which schools were eligible for LCRSG if more than 97 percent of their students were deemed as high need to calculate the effect of the program in a regression discontinuity design. To enhance power and account for covariate imbalance, this paper also employs a two-way fixed effects design with bandwidth restrictions to compare ineligible schools to eligible schools over time. Estimates are computed at multiple bandwidths graphically and statistically. This paper also includes document analysis on annual reports written by school leaders and queried from the California Department of Education through a Freedom of Information Act request. The author reads through this corpus of documents twice and codes them using both inductive and deductive codes to identify themes. 

Using a two-way fixed effects model with a bandwidth of 0.25 standard deviations (SDs) around the cutoff for LCRSG eligibility, this paper finds that LCRSG improved the average grade-3 English Language Arts score by 0.05 SDs across its first two years. This effect size estimate is similar when calculated using the bandwidth selection procedure of Calonico et al. (2014). When decomposed by year though, this program had a large effect (0.09 SDs) in year two and a very small effect in year one. 

An examination of school staff reflections shows that few funds were spent in the first year as schools planned. In the second year, fifty-eight percent of school sites used the funds to hire literacy coaches while 30 percent provided student intervention services, like tutoring, to low-performing students. School sites reported minimal staff resistance and relatively high levels of buy-in, thanks in part to the provision of funds that accompanied professional development about how to teach. 

Author

• Sarah Novicoff, Public Policy Institute of California
  Presenting Author