BackHome
Refresh: Off
Paper Summary
Share...

Direct link:

Implementation and Impact of a No-Cost District Summer School Program

Thu, April 9, 2:15 to 3:45pm PDT (2:15 to 3:45pm PDT), Los Angeles Convention Center, Floor: Level Two, Room 409AB

Abstract

Objective

This study examines the barriers to enrollment and participation in a mid-sized urban district’s no-cost summer learning program and explores how these barriers intersect academic and social-emotional impact.

Perspective

Guided by a utilization-focused framework (Patton, 2008), this mixed-methods study draws on two years of data to inform district decisions regarding future programming by exploring how structural, logistical, and perceptual barriers shape participation and outcomes in a district-run summer program

Methods

The study used a quasi-experimental design with inverse probability of treatment weighting to estimate effects. Outcomes included math and reading/ELA Classroom Diagnostic Tools (CDT) scores (Y1, Y2), and social-emotional survey scores (Y1). Effects were explored across attendance dosages. Qualitative data were analyzed to define barriers, understand implementation, contextualize quantitative findings, and inform recommendations. All activities were conducted in collaboration with district staff.

Data Sources

Quantitative data included district administrative records on enrollment, attendance, demographics, staffing, cost, SEL surveys, and assessments. Qualitative data included program documents, classrooms observations, student focus groups, staff interviews, and surveys.

Results

While the program was free for district students and widely accessible, multiple barriers limited consistent participation.

Families viewed the program as a backup to more appealing summer options.

Families perceived the program as optional and there were no consequences for students’ non-attendance.

Limited enrichment options and unengaging academic content influenced sustained student participation.

Site selection based on air conditioning availability led to increased travel distances and transportation challenges.

Limited coordination with the Special Education Department led to barriers for students with IEPs. Some enrichment providers lacked training to support students with IEPs.

Quasi-experimental Y1 outcomes analyses showed lack of meaningful effects in math and reading/ELA performance. Social-emotional effects related to emotional regulation in Y1 were promising (ES=.09, p<.05), particularly in emotional regulation for boys (ES=.14, p<.05).

In Y2, students who attended 20+ days showed gains in math (ES=.06, p<.10), whereas there was no meaningful effect for those attending <20 days. There were no meaningful effects in reading in Y2. Moderator analyses revealed negative effects for students with IEPs.

During summer 2024 site visits, observers found that the class sizes were less than half of the target class size of 30 students, with an average of 11.6 students observed in attendance across classes; the program was overstaffed.

Significance

Evidence documenting the impacts of summer programs on student achievement is mixed. Two meta-analyses show impacts on achievement in math (Lynch & Mancenido, 2023) but no measurable impacts on ELA (Augustine et al., 2016). Conversely, a large evaluation of an eight-city program found impacts in math and ELA, and linked outcomes to balanced academic and enrichment time and high attendance (Borman et al., 2024). Building on extant literature and study results, researchers recommend: 1) Clearly defining the program’s primary goal (e.g., academic support vs. summer care), 2) Requiring daily attendance tracking, 3) Changing attendance policies to support sustained participation, 3) Beginning planning and training in January (Diliberti & Schwartz, 2024), 4) Tracking interests in enrichment activities, and 5) Balancing academic and enrichment time.

Authors