Assessing the Impact of an RPP on a Large US School District: The Case of CAFÉCS

• In Event: Assessing and Monitoring the Effectiveness of Research Practice Partnerships: Insights From International Experiences

Abstract

1. Objectives
This study examines one long-standing partnership’s approach to documenting its impact in a large urban school district in the United States. We describe the collaborative development of a framework to assess how the partnership supported improvements to district 1) Programs, 2) Research, 3) Organizational Structures, and 4) Policies leading to 5) Equitable Results for students. This framework was used to document partnership impact across these areas, drawing on 11 years of RPP-generated dissemination products, grant proposals, evaluation findings, and information-gathering sessions with the leadership team. Through this analysis, we aim to support RPPs in the complicated but much-needed process of assessing partnership impact.
2. Perspective(s) or Theoretical Framework
Assessing RPP impact and progress towards partnership outcomes was also identified early on as an essential aspect of the work (Henrick et al., 2017). The majority of RPP research to date generally explores RPP processes and ways of working and does not attempt to study RPP outcomes (Schneider, 2020) or answer the question: “What did the partnerships change, not just in terms of research use or service delivery, but in what matters the most, which is improved outcomes for students” (Schneider, 2018).

3. Data Sources & Methods
After conceptualizing the PROSPER Framework, the authors engaged in a systematic review of project documents. Through an audit of the partnership’s work between 2011 and 2022, authors identified ten grants awarded to the partnership, and 34 dissemination products between 2015 and 2022. The full set of artifacts were read and analyzed by four of the study’s authors using deductive coding (Creswell & Creswell, 2017) to document evidence of district impact around each of the five PROSPER Framework components in a spreadsheet. The results from the coding process were synthesized using thematic analysis (Creswell & Guetterman, 2019).
4. Findings & Conclusions
Broadly speaking, this analysis indicated that the CAFÉCS RPP impacted the district across each of the five elements of PROSPER:
1) Programs: CAFÉCS impacted the selection and implementation of CS programs;
2) Research: CAFÉCS conducted research to understand and address problems of practice facing the OCS;
3) Organizational Structure: CAFÉCS funding impacted the OCS budget, resulting in changes to organizational structures in the OCS;
4) Policy: CAFÉCS impacted the enactment and implementation of district policy; and
5) the work of CAFÉCS led to Equitable Results in CS education in CPS by expanding capacity to offer CS equitably across the district, increasing equitable access to CS, increasing equitable participation in CS, and ensuring equitable outcomes related to student experiences in CS.

5. Significance of the Study
While the identification and categorization of different areas of impact was helpful for annual reporting and strategic planning, we believe the PROSPER Framework’s greater value lies in providing a lens for RPPs to examine how these areas of impact can inform, interact with, and guide one another. We consider this analysis to be one step toward understanding the complexities of RPP district impact and hope that this framework can be used by other RPPs to consider their impact in other district contexts.

Authors

• Erin C. Henrick, Vanderbilt University

• Daniel Schmidt, Partner to Improve

• Steven McGee, The Learning Partnership

• Andrew Rasmussen, Chicago Public Schools

• Lucia Dettori, DePaul University

• Ronald I. Greenberg, Loyola University Chicago

• Dale F. Reed, University of Illinois at Chicago

• Don Yanek, DePaul University