Disentangling COVID-19 Learning Loss from Schooling Loss

• In Event: The impact of COVID on child development and learning loss

Abstract

In early 2020, school closures were implemented around the world to curb the spread of COVID-19, leading to concerns about consequential “learning loss” in school-age children (e.g., Engzell et al., 2021; Hallin et al., 2022). Yet whether children actually experienced a decline in knowledge beyond their disrupted schooling experience is an empirical question. In this paper, we use norm-referenced tests of literacy and math skills to disentangle COVID-19 learning loss (i.e., a decline in knowledge or skills) from schooling loss (i.e., losing ground compared to prior cohorts of age-matched peers who had never experienced school closure).

Using data from a larger study examining the long-term impacts of early education, we followed 625 children from Tulsa, Oklahoma (91% from low-income homes; 50% female; 55% Hispanic/Latinx, 18% Black, 11% White, 9% multiracial, 6% another race/ethnicity) from 2018-2021 (though data collection was paused from spring 2020-spring 2021 due to school closures). Children’s literacy and math skills were assessed in the fall and spring of kindergarten (2018-2019), fall of first grade (2019), and fall of third grade (2021) using the Woodcock-Johnson III Letter-Word Identification and Applied Problems subtests (Woodcock et al., 2001). Both raw scores (the number of correct items responded between the basal and ceiling items) and standard scores (raw scores transformed based on pre-COVID age-stratified norming samples with a population mean of 100 and a standard deviation of 15) were collected. The availability of both raw and standard scores allows for an exploration of whether children are in fact experiencing learning or schooling loss. If it is indeed a learning loss, then both raw and standard scores should decrease post-school closure. Decreases only in standard scores, however, would indicate schooling loss.

Raw and standard scores were analyzed using repeated-measures ANOVAs using jamovi (version 2.2). Means and confidence intervals for literacy and math are reported in Figure 1a) and b) respectively. On average, children’s raw literacy scores significantly increased from each wave to the next, F(3,1872)=2834, p<.001, Eta2=0.531. Similar patterns of results were observed in raw math scores, F(3,1872)=1824, p<.001, Eta2=0.411. For both literacy and math standard scores, children’s performance differed across time, Fs(3,1872) > 106, ps<.001, Eta2-Literacy = .035 and Eta2-Math = .121. For literacy, standard scores significantly increased in the spring of kindergarten relative to the fall of kindergarten, but decreased from spring of kindergarten to fall of first grade, possibly due to summer learning loss. Standard scores in the fall of third grade, post-COVID, were the lowest of the study period. For math, standard scores increased from the fall to spring of kindergarten, but scores between the spring of kindergarten and fall of first grade were not statistically different. As with literacy, standard math scores post-COVID were lower than all previous waves.

Findings reveal that learning loss was only observed in standard score comparisons against a sample of children who had never experienced school closure, and children actually displayed growth in literacy and math raw scores from kindergarten to third grade, indicating a lower rate of learning associated specifically with a loss in schooling.

Group Authors

Tulsa SEED Study Team

Authors

• Gigi Luk, McGill University
  Presenting Author

• Anne Partika, Georgetown University

• Anna D. Johnson, Georgetown University

• Anne Martin, Georgetown University

• Diane M. Horm, University of Oklahoma-Tulsa