Fidelity and Structured Teacher Adaptations: Evidence from an Online Content Literacy Intervention During COVID-19

• In Event: Investigating Teacher Knowledge and Adaptability: Enhancing Literacy Outcomes in K-3 Students

Abstract

Objectives

We examine the effects of structured teacher adaptations to a content literacy intervention delivered online during COVID-19 school closures. We randomly assigned 95 classrooms to either Core Treatment, replicating prior implementations (e.g., Kim et al., 2017), or Adaptive Treatment, which enhanced the core treatment with structured teacher adaptations. Our research questions are: What are the effects of Adaptive Treatment compared to Core Treatment on student engagement in asynchronous activities (RQ1), student-teacher interactions during synchronous lessons (RQ2), and student outcomes in science vocabulary, background knowledge, and reading comprehension (RQ3)?

Theoretical Framework

Two dimensions of program implementation—fidelity and adaptation—impose distinct demands on teachers in evidence-based interventions. The fidelity-focused model expects teachers’ exact delivery of the program, viewing deviations as compromising the program’s integrity (Domitrovich et al., 2010). Conversely, the adaptive approach permits modifications that maintain core components while adjusting to local needs (Bryk et al., 2015). Fidelity and adaptation, when viewed as synergistic rather than competing priorities, enhance program implementation and impact (Durlak & DuPre, 2008; Kim et al., 2017). Structured teacher adaptations provide a systematic framework that supports program modifications without compromising core principles, increasing relevance, applicability, and effectiveness. This approach integrates the strengths of both experimental science (high fidelity) and improvement science (high adaptation), effectively bridging the research-practice divide (Kim & Mosher, 2023).

Methods

The participants were 2,247 third graders and 95 classroom teachers from 26 elementary schools. Both Core and Adaptive Treatment teachers delivered the same core components of the intervention through asynchronous (app and books/trifolds) and synchronous Zoom lesson activities in a five-week “human body system” unit. Adaptive Treatment teachers further implemented structured adaptations and collaborated with the research team to develop and apply strategies to enhance student engagement.

Data Sources

Data sources included a teacher survey on fidelity of implementation, classroom observations evaluating the quality of student-teacher interactions (engagement, questioning, feedback), and assessments of student learning including depth of science vocabulary knowledge through a semantic association task, science content reading comprehension and background knowledge, reading comprehension (Measure of Academic Progress (MAP) and End-of-Grade (EOG) assessments), and backend app data tracking student engagement.

Results

RQ1. The Adaptive Treatment group exhibited statistically higher engagement in app library usage, science book completion, and targeted word learning (ps< .05). This treatment also improved motivational engagement, with students showing greater enjoyment and perceived self-competence.
RQ2. Adaptive Treatment significantly improved the quality of student-teacher interactions.
RQ3. Adaptive Treatment notably improved students’ science background knowledge (ES=.09) and science content comprehension (ES=.07). No significant effects were observed on science vocabulary knowledge depth and reading comprehension.

Scholarly Significance of the Study

This study demonstrates that structured teacher adaptations enable effective modification and extension of evidence-based programs, amplifying engagement and learning outcomes while maintaining negligible treatment heterogeneity across schools. It underscores the synergistic potential of fidelity and adaptations in interventions, enabling educators to bridge the gap between research-based intervention and classroom dynamics. This approach broadens the impact of programs, improving their relevance and effectiveness to meet the evolving needs of learners and educators.

Authors

• Jackie E. Relyea, North Carolina State University

• Joshua Baer Gilbert, Harvard University

• Mary Burkhauser, Harvard University

• Ethan Scherer, Harvard University

• Douglas M. Mosher, Harvard University

• Zhongyu Wei, Harvard University

• Johanna Tvedt, Harvard University

• James S. Kim, Harvard University