Effectiveness Analyses Associated with the Chicago Community Schools Initiative

• In Event: Evaluating the Impact of Community Schooling: Comparing Methodologies and Findings Across States

Abstract

Objectives or Purposes
Since 1998, Chicago Public Schools (CPS) has implemented the Community Schools Initiative (CSI) to provide supports and opportunities for students, families, and the broader school community to promote positive youth development. This study summarizes a series of effectiveness analyses evaluating whether CSI participation and funding are associated with improved student outcomes. Specifically, the analyses addressed two questions:
What effect did sustained participation in CSI programming have on student outcomes, compared to similar students either (a) attending CSI schools but not participating, or (b) enrolled in matched non-CSI schools?
What effect did CSI funding have on student outcomes compared to similar schools that did not receive CSI funding?
Theoretical Framework
Anchored in the CSI implementation framework (Zander et al., 2010), the initiative is grounded in evidence-based strategies that support whole-child development and positive family and community engagement. The framework includes:
Cross-sector partnerships with community agencies and providers
Enhanced coordination of supports and opportunities within schools
Multiple pathways for collaborative leadership and authentic community and family voice
Methods and Data
Six separate CSI effectiveness analyses were conducted using a range of research designs and methodologies, including difference-in-differences and comparative interrupted time series designs, as well as the use of propensity score matching to select comparison students and schools. These approaches enabled rigorous comparisons of outcomes while addressing selection bias and variation in implementation.
Outcomes examined in these analyses included academic performance (e.g., reading and mathematics scores on state and formative assessments, grades), school attendance, disciplinary incidents, and school climate as measured by the 5Essentials survey. Data were obtained from district data warehouses and online systems designed to track CSI activity and service participation.
Results and Conclusions
Overall, these analyses demonstrated that implementation of the CSI strategy has been associated with positive effects related to improved school-day attendance, fewer disciplinary incidents, and improved student perceptions of school climate and culture (see Appendix C). The magnitude of these effects ranged from small to large. Evidence of academic achievement gains was more limited (with most effects related to grade point average) and, in the case of assessments, emerged primarily in schools that prioritized academic supports as part of afterschool programming and where a significant portion of students consistently participated. There was also evidence of an association between a higher level of participation in CSI programming and larger effects in many of these areas, including academic achievement.
Scholarly Significance
This body of work offers a sustained assessment of a long-standing community school initiative in a large urban setting. The evaluations provide insight into the promise and limitations of CSI and demonstrate how program effectiveness varies with levels of participation and alignment with core supports.

Authors

• Neil Naftzger, American Institutes for Research

• Dominique Bradley, American Institutes for Research