Using Quasi-Experimental Methods to Measure the Long-Term Academic Impacts of Community School Initiatives

• In Event: Evaluating the Impact of Community Schooling: Comparing Methodologies and Findings Across States

Abstract

Purpose
As community schools (CS) gain momentum as a strategy for advancing educational equity, questions remain about their long-term effectiveness on student outcomes (Heers et al., 2016). Despite widespread adoption and federal investment (Blank et al., 2023), rigorous longitudinal studies remain scarce (Prenger et al., 2022). Research and evaluation models must capture both academic and non-academic outcomes over time to reflect the holistic vision CS advances. This study demonstrates how rigorous quasi-experimental designs supported by carefully defined sample inclusion criteria and propensity score matching can produce credible evidence. Quasi-experimental methods are widely used in educational research to estimate treatment effects (Cook & Campbell, 1979), especially when randomized controlled trials are unfeasible or ethically problematic (Slavin, 2002). Therefore, this study examined whether the impacts of elementary CS programming extended into students’ middle school performance on college readiness indicators compared to a demographically matched group of students who did not attend CS.
Perspectives
The equity-driven, whole-child framework guiding this work recognizes that structural barriers (e.g., poverty, racism, and under-resourced schools), require comprehensive, place-based responses (Jacobson, 2016; Maier et al., 2017). Community schools embody this approach by integrating academics with health, social, and enrichment services (Min et al., 2017). CS services are integrated, making it difficult to decipher which mechanisms are most influential on education outcomes. This complexity presents methodological challenges that cannot be overlooked.
Methodology
A quasi-experimental approach measured how CS supports translate into measurable academic gains over time. Propensity score matching (Handley et al., 2018) was utilized to compare outcomes of demographically and academically similar CS and non-CS students.
The sample included 108 students (half attended the same CS for at least three years during their elementary education and half attended a non-CS elementary school). All students subsequently attended non-CS middle schools in the same district. Data sources included school-maintained academic records, with GPA and course failures analyzed as indicators of academic progress, dropout risk, and college readiness. Repeated measures ANOVA and Fisher’s exact tests were used to examine changes in GPA over time and group differences in risk and readiness outcomes.
Results
Results revealed former CS students experienced statistically significant GPA gains across middle school, while matched peers did not. By eighth grade, students in the CS group were significantly more likely to have avoided course failures, maintained a GPA above 2.0, and met thresholds associated with future college access. No students in the comparison group met the most selective GPA benchmark (≥3.5), while 13% of CS students did.
Scholarly Significance
This study offers a replicable model for evaluating complex CS initiatives. It demonstrates how quasi-experimental methods when designed with care and contextualized through research-practice partnerships can yield actionable findings. The implications are timely given the rise in CS funding and the need for evidence-based accountability frameworks. This work demonstrates the value of CS as both a school transformation strategy and a research priority. It supports continued investment in integrated, equity-oriented education approaches and calls for further research to track their impact on long-term student success.

Authors

• Kathleen Provinzano, Binghamton University - SUNY

• Toni A. May, Binghamton University - SUNY

• Naorah Rimkunas, Binghamton University - SUNY