Increased Collaboration and Alignment in Out-of-School Time Programs: Formative/Summative Evaluation Results

• In Event: Leveraging Partnerships to Improve Outcomes for English Language Learners: Designing, Implementing, and Evaluating Effective Out-of-School Time Programs

Abstract

Content and Objectives: This paper reports on two years of data gathered on key demographic indices, achievement scores, and survey data from program participants (students & staff). We report on the three OST programs that were part of our project as well as comparison data on similar programs in the city. By reporting on program data we were able to demonstrate positive effects of participation in our project that were realized by students and staff. We report on specific outcomes related to ELL students. Through multi-agency collaboration and engaging multiple stake holders (students, parents, teachers, administrators, community organizations, IHEs, and program evaluators) we were able to accomplish our program goals of collaboration and alignment to improve ELL outcomes in some areas, but not in others.

Conceptual framework: Two theoretical frameworks were used to guide our research: 1) ecological systems approach (Durlak, Mahoney, Bohner, & Parente, 2010); 2) capitalizing on funds of knowledge (Moll, Amanti, Neff, & Gonzalez, 1992). When schools are viewed as ecological systems where the actors engaged in reciprocal relationships, shifts occur in attitudes, behaviors, knowledge, and skills. Our theory of change assumes that through intentional integration of OST program educators with regular school day program teachers, alignment of the curriculum, engagement of parents, and collaboration across stakeholders, a functioning system allows for students are expected to experience greater benefits. In addition, we perceived the educators to bring with them rich knowledge and skills drawn from their lived experiences in the communities, as well as linguistic and cultural capital that allowed them to connect with and include their students in the curricula.
Data: Two years of data were analyzed to identify key differences between participants in our OST programs and other programs. Similar to data drawn from existing evaluation reports for the previous two papers, data for this paper is more comprehensive in reporting overall demographics across all three sites, as well as key findings from student and teacher surveys about background, attitudes, and behaviors.
Findings & Conclusions:
•Academic Performance – Report Cards
•At all three sites, participants improved their math and ELA grades over the course of the year, at rates on par with or exceeding similar 21C programs in NYC.
• Student Responses
• The majority of youth indicated plans to graduate from college or obtain an advanced graduate degree.
• Youth held generally positive attitudes about their community, which got stronger following program participation.
• Activity Observations
• Youth were engaged in robust activities that required analytic thinking and offered a strong intellectual challenge, facilitating their creative and academic development.
While the demand for OST programs has increased dramatically in recent years, the research and guidance on effective models is scarce. This paper provides a comprehensive overview of design, implementation, and evaluation of an effective OST program.

Authors

• Shannon Stagman, ExpandED Schools

• Rachel Chase