Factors Influencing Pandemic Learning Disruption and Recovery: Evidence from California

• In Event: Supporting Students Through Educational Disruptions: State and District Strategies for Post-Pandemic Education Renewal

Abstract

1. Objectives
The presentation summarizes work that examines changes in achievement and attendance during the pandemic (2019-22) and the initial period of learning recovery (2022-23 and 2023-24). Using student-level data for all California public schools along with school demographics, district characteristics, and a rich array of measures of COVID recovery plans at the district level, the presentation documents learning loss and recovery, overall and by student subgroup, and relates these changes to school and district policies.

2. Theoretical framework
The research adopts the framework that changes to achievement during the pandemic reflected many forces outside of education, and that the pandemic likely affected socioemotional learning.

3. Methods
The research takes a regression-based approach. In the first stage, learning loss between 2019 and 2022 is estimated at the school level, along with learning recovery between 2022 and 2024. Student and school characteristics are included so as to minimize omitted variable bias. In the second stage, each school's pattern of achievement change is modeled as a function of school and district characteristics, and policies implemented during the pandemic to assist students.

4. Data
Data include administrative data for all California public school students, focusing on attendance, grades, courses taken and test scores. Two unique sources of data are merged with the student level data: 1) A statewide survey of public school districts' policies during the pandemic and 2) a unique database that was created by analysis of California districts' plans for spending funds from the Expanded Learning Opportunities Grants (ELO-G). A fourth source of data -- the California Healthy Kids Survey (CHKS) -- provides a school-level portrait over time of school climate and student engagement. The final dataset includes numerous other measures including overall COVID recovery spending per pupil by district, and school re-opening dates.

5. Results
The analysis of school-level trends in achievement, after taking into account changes in the student composition of schools, shows widespread drops in achievement between 2019 and 2022, with bigger losses for math than English Language Arts. Schools varied dramatically, with most experiencing drops in achievement, but with some showing gains. Recovery between 2022 and 2023 was partial, and extremely uneven. Drops in achievement are notably correlated with drops in attendance rates. COVID expenditures to help students recover academically are associated with gains in achievement. Similarly, schools that re-opened earlier experienced smaller learning losses. But in both cases, these policy-induced gains were small relative to overall initial losses. Ongoing work, that will be complete before AERA, is studying the relation between specific district recovery policies and learning recovery in 2022-23 and later, including factors such as extended learning time and its mode, socioemotional supports for students, professional development for teachers, tutoring, and its specific form, and many other factors.

6. Significance
This is one of the first pandemic studies that combines detailed student-level data on a wide variety of student outcomes with district survey data and information from district recovery plans to infer patterns of loss and recovery.

Authors

• Julian Betts, University of California - San Diego

• Niu Gao, American Institutes for Research

• Jonathan Isler, California Department of Education

• Bruce Fuller, University of California - Berkeley