When Evidence Isn't Enough: the Grammar of Schooling and the Limits to Educational Recovery Strategies During COVID-19 in the United States

• In Event: Reforming Education at Scale: Bureaucratic Structures, Policy Ambitions, and Systemic Constraints in Comparative Perspective

Proposal

Purpose
This exploratory study seeks to develop a deeper understanding of the institutional forces that make it difficult to sustain and scale even so-called “evidence-based” educational improvement strategies. To accomplish this goal, we analyze the main policies passed by the US Congress to support schools during the COVID-19 pandemic and examine the use and implementation of high-dosage tutoring as a recovery strategy. We aim to make visible how some of the same institutional incentives and capacity constraints that sustained the “grammar of schooling” (Tyack & Cuban, 1995) in the 20th Century continue to limit the development and implementation of educational policies in the 21st Century – even those that appear to have some empirical support, political consensus, and substantial funding.

Theoretical Framework
This study draws on a theoretical perspective that explains how institutional, political, and societal forces sustain the “grammar of schooling” (Tyack & Cuban, 1995). The grammar of schooling describes the “egg-crate” structure separating students into different age levels and classrooms to learn discrete subjects from individual teachers, which continues to dominate conventional schooling around the world. The literature on the persistence of the grammar of schooling highlights that although reform efforts seek to change schools, schools also change reforms, forcing new policies and reform initiatives to adapt to the status quo (Author et. al., 2021; Cohen & Mehta, 2017). This process produces hybrids of new and conventional practices that contribute to more incremental changes than many reform efforts promise.

Methods & Data Sources
We build on a larger study of research and policy documents that discuss the development and implementation of international educational responses to the COVID-19 pandemic and their implications for educational transformation. For this analysis, we focus on the US and trace policy reaction to the pandemic through government documents, including federal education recovery legislation (CARES Act, CRRSA Act, American Rescue Plan) and U.S. DOE reports. We highlight the development and implementation of an “evidence-based” recovery strategy – high-dosage tutoring – because it is prominently featured in legislation as a “recovery” strategy for which there is some evidence of effectiveness.

Results
The efforts to sustain learning during the school closures and to implement high-dosage tutoring to support learning recovery illustrate that even when there is some information about “what works,” contextual differences, institutional factors, and capacity challenges limit implementation and scale-up. As the authors of OECD’s early assessment of education system responses to the pandemic state: “In times of emergency, speed in the implementation of responses is key, but evidence of what may work is limited, and constraints on resources and capacity are binding” (Gouëdard et al., 2020). Within this context, even if everyone agreed on “what works,” the diversity of values and beliefs about the goals of education and the political decision-making process make it difficult to come to a consensus on how much to invest and how to take research findings into account (Cairney, 2017).
The research consensus around high-dosage tutoring helped to inspire a host of private initiatives to make tutoring available on a large scale. The Gates Foundation, Arnold Ventures, and the Overdeck Family Foundation, for example, established Accelerate, a new organization with a mission to “embed high-impact tutoring programs into public schools now and for the long term” (Accelerate, 2023). By 2025, Accelerate reported leveraging “over $300 million in federal and state public funding for proven personalized learning interventions” (Accelerate, n.d.).
With this research consensus, public and private support, and substantial funding, tutoring spread rapidly across US schools as a recovery strategy. What happened? First, in addition to evidence of effectiveness, the spread of tutoring benefits from its ability to fit into and conform to many aspects of the grammar of schooling (Author, 2023). Tutoring focuses on conventional academic subjects and outcomes; it can be assessed with tests in widespread use; and it can be deployed within conventional classes and schedules or outside of school as a complement to conventional instruction. Tutoring also looks like “real school” (Metz, 1989) as teachers and expert adults provide instruction and guidance focused on academic content. Furthermore, tutoring lent itself well to online and hybrid settings and helped schools to fulfill the societal need for childcare during the school closures by keeping children occupied online in “Zoom schools.” These factors made it easier to put tutoring in place than to explore more innovative practices that could be more disruptive.
At the same time, previous experiences with large-scale reform efforts anticipated several predictable capacity problems that limited the ability of schools to meet the criteria for high-dosage tutoring. Critical capacity issues in this case included a lack of trained personnel and difficulty ensuring that those needing high-dosage tutoring actually got it. Shortages of teachers meant that many tutoring plans had to scale back significantly (Carbonari et al., 2024). Furthermore, even with significant incentives to participate in tutoring programs, students did not enroll or did not show up (Robinson et al., 2022). These capacity issues and other challenges contribute to a well-known pattern of declining effects whenever educational initiatives attempt to scale (Makori et al., 2024; White et al., 2023; Kraft et al., 2024).

Conclusion & Significance
This paper addresses a fundamental challenge in school reform by explaining critical challenges that limit the reach and effectiveness of even “evidence-based” recovery strategies. Consensus on “what works” does not enable educators to overcome conventional pressures that support the status quo and encourage schools to add onto rather than change the practices and structures of conventional schooling. This analysis suggests a shift from debating whether or not a strategy or investment “works” to a broader consideration of “what can work” that takes into account 1) the evidence available; 2) the sources of data and criteria for determining value and “effectiveness;” 3) the goals and assumptions for education and education initiatives, and 4) the “fit” with conventional structures and expectations and the time, resources, and expertise available.

Authors

• Thomas C Hatch, Teachers College, Columbia University

• Jonathan Beltran, Teachers College at Columbia University