Lessons From the Pandemic: The Effects of Remote Instruction and Recovery Strategies on Student Outcomes

• In Event: Roads to Recovery: State and District Approaches to Post-Pandemic Education Renewal

Abstract

Objectives. The COVID-19 pandemic and rapid shift to remote learning has had profound effects on K-12 education in the U.S. and continues to impact teaching and learning. Student learning trajectories have been negatively affected, which may exacerbate long-standing disparities in student achievement and worsen longer-run student outcomes. Aided by federal funding from the American Rescue Plan Act, state education agencies and local school districts are employing a variety of strategies to accelerate student learning and promote student recovery from the negative effects of the pandemic, yet there is great uncertainty as to what programs will be most effective and for whom. Working in partnership with three Atlanta-metro area school districts, we conducted evaluations of several recovery strategies including high-dosage tutoring, extended school days, and summer programming.

Theoretical framework. Working within the improvement sciences tradition and in close partnership with district partners, our study aims to extend understanding of local implementation processes, nuanced understanding of what has the potential to work for whom, and the scaling of rigorous evidence-based practices (Joyce & Cartwright, 2020; Lewis, 2015).

Data sources. District partners provided log data from individual tutoring and virtual learning sessions as well as student-level program eligibility, formative assessment (iReady and MAP growth), and sociodemographic data. Data were provided for the 2020-21 and 2021-22 school years. In addition, the districts have administered a parent survey to students’ experiences during and after the COVID-19 pandemic, including variation over the course of the pandemic in terms of engagement, autonomy, competence, relatedness, and COVID-19 related life events.

Methods. We calculated descriptive statistics to examine student usage of recovery programs. We subsequently ran multivariate regression models to examine associations between student usage and achievement growth controlling for baseline achievement and student characteristics likely to be associated with participation and/or subsequent outcomes.

Results. The return to near-universal in-person learning in SY 2021–22 did not yield substantial improvements in average math or reading achievement growth. While some students are now at or near their pre-pandemic achievement levels, many other students remain substantially below their pre-pandemic achievement levels. Despite this, we observed low rates of student participation in optional summer school and virtual tutoring programs. Students invited to participate in summer school achieved comparable achievement gains as students not invited. However, a small group of intensive tutoring users appear to have benefited from the impact estimates are imprecise. In addition, prior to AERA, we intend to have analyses ready to share documenting students’ holistic experiences during and after the pandemic as well as more detailed and longer-term evaluation of recovery strategies.

Significance of the study. Understanding how well initial recovery strategies are working and for whom will allow districts to make “mid-course corrections” during the multi-year recovery period, such as working to improve student participation in potentially helpful recovery interventions. Understanding how well students are responding to acceleration initiatives will also provide information on longer-run impacts that may need to be addressed beyond the initial recovery period.

Authors

• Jennifer Suzanne Darling-Aduana, Georgia State University

• Tim Sass, Georgia State University

• Salma Mohammad Ali, Georgia State University

• Sarah Sterling King, Christopher Newport University

• Thomas Goldring, Georgia State University