Evaluation of the Networks for School Improvement Initiative—Relationship Between Network Hub Strategies and Supports and School Implementation of CI Activities

• In Event: Networked Continuous Improvement at Scale: Research Findings and Reflections from the NSI Initiative

Abstract

Networked continuous improvement has grown substantially in popularity in districts and schools over the past decade and is a now common approach to improvement (Anderson et al., 2024; Sandoval & Van Es, 2021). The Gates Foundation’s Networks for School Improvement (NSI) initiative presented one of the first opportunities to study the use of networked continuous improvement at scale, in part because the foundation specifically designed the evaluation to measure implementation and impacts across the NSI rather than evaluating individual NSI. The summative evaluation of the NSI initiative involved researchers from several thinktanks and research organizations working collaboratively to answer three primary questions:
1. How do intermediaries design and implement their NSI?
2. To what extent do participating schools implement CI activities?
3. What is the impact of the NSI on student outcomes?
In this paper, researchers from AIR and RAND identified strategies and contexts of intermediaries and networks that are associated with networks that change educator and/or school practice. Utilizing five years of data collected by AIR and RAND from NSI network hubs and their schools, we explore the relationship between network hub strategies and contexts, the cohesiveness of their networks, and implementation of continuous improvement activities in schools.
We draw upon two main sources of data. RAND administered the Team Connections Survey (TCS) to school team leaders in 25 networks over the course of five years, beginning in the spring of SY 2020-21 through SY 2024-25. This social network survey included questions covering the supports provided by network hubs to schools (and the perceived quality of these supports), interactions between the network hub and network schools and among network schools, and perceived benefits of network participation. For these same years, AIR collected extensive artifacts -- documents generated during CI for about 500 schools working with 25 NSI participating in the initiative. These documents were analyzed to understand the level of implementation of continuous improvement activities in the NSI schools.
Together, we use measures from the TCS to predict implementation of CI activities, using hierarchical linear modeling (HLM) to account for the nested structure of the data (i.e., grant years, nested in schools, nested in networks). We analyze the relationship between the focal variables of interest (e.g., strength of coaching) and the outcome (e.g., number of CI cycles completed). We also assess sources of variation, such as whether the network contributes to changing school practices (i.e., the variation is mostly across networks) rather than differences in the schools within a network (i.e., the variation is mostly across schools within a network).
The findings from this analysis provide unique insights into the ways in which common network hub-level strategies and supports contribute to school-level implementation of continuous improvement across a wide range of network and school contexts. These insights may help inform intermediaries, districts, and school decisions about the supports and methods to enhance and support their continuous improvement work.

Authors

• Michael S. Garet, American Institutes for Research

• Matthew J. Farmer, American Institutes for Research

• Ryan C. Eisner, American Institutes for Research

• Stephani L. Wrabel, University of Southern California

• Becki Herman, RAND Corporation

• Susan Bush-Mecenas, RAND Corporation