Findings From the Implementation Study of the Reading Recovery Investing in Innovation Fund (i3) Scale-Up

• In Event: Final-Year Results From the External Evaluation of the Reading Recovery Investing in Innovation Fund (i3) Scale-Up

Abstract

The i3-funded independent evaluation of Reading Recovery included a comprehensive, mixed-methods implementation study that included three distinct foci: implementation fidelity; lesson-level implementation; and school-level implementation of Reading Recovery. The second and third foci emerged in the first two years of the project, as CPRE researchers concluded that a sole focus on fidelity could not tell the whole story of what matters about Reading Recovery’s implementation.

The design of the implementation study evolved in response to the emergence of these new foci. In Year One, the qualitative and quantitative strands of the research were conducted in parallel. In Year Two, our approach shifted toward an exploratory instrument-development approach (Creswell, 2013; Creswell & Plano Clark, 2007) in which qualitative data collection preceded survey administration in each annual research cycle. This enabled us to explore emerging questions in depth and later study those questions at scale via survey. In Years Three and Four, we combined our qualitative and quantitative efforts in a fully integrated mixed design (Teddlie & Tashakkori, 2009) that was iterative in nature and that assigned equal priority to qualitative and quantitative methods (Hall & Howard, 2008). The strategic sequencing of the qualitative and quantitative methods added rigor to our analysis, and the multi-source data produced deeply contextualized findings that demonstrate how the three critical factors for instruction operate across a range of contexts, and through the involvement of multiple actors (Ravitch & Carl, 2016).

Key findings from the three stands of the implementation study include the following:

• Schools in the i3 scale-up implemented Reading Recovery with high fidelity to the program model, as codified in the Standards and Guidelines of Reading Recovery in the United States, 6th Ed. (Reading Recovery Council of North America, 2012) in all four years of the study. This finding supports the validity and interpretability of our experimental findings.

• Three aspects of the program’s implementation are critically related to the effectiveness of one-to-one lessons at facilitating student progress. These are:
(1) The strength of the teacher delivering the lessons. In presenting our findings on this issue, we offer a novel conceptualization of teacher strength in Reading Recovery.
(2) The extent to which school principals prioritize Reading Recovery lessons.
(3) District-level protection of one-to-one lessons, and troubleshooting of threats.

• School-level implementation of Reading Recovery can be understood through four schemas of implementation: isolation, obstruction, endorsement, and integration. Schools’ manifestations of these schemas are indicative of both the sustainability of their Reading Recovery programs and the impact of the intervention on students not directly served by one-to-one lessons.

Along with describing these findings and the associated research methods, this paper will discuss the contributions of the implementation study to scholarship on Reading Recovery; on teacher quality; on leadership of instructional programs; of the impact of context on implementation; and, more broadly, on the conduct of implementation research in the context of experimental studies.

Authors

• Abigail Gray, University of Pennsylvania

• Heather D Goldsworthy, University of Pennsylvania

• Helen Anderson-Clark, University of Pennsylvania

• AJ Schiera, University of Pennsylvania

• Toscha J Blalock, University of Pennsylvania