Findings From a Randomized Controlled Trial on Five Voluntary Summer Learning Programs Across the Country

• In Event: 52.044 - Recent Research Insights Into Summer Learning Losses and Gains

Abstract

The goal of this study is to determine the effectiveness of voluntary summer learning programs by comparing students who participate in these programs to similar students who do not. The study was conducted in five school districts located in Boston, Dallas, Duval County, FL, Pittsburgh, and Rochester, NY. From a pool of student applicants, a lottery was used to select students to either participate in the program or serve in a control group. The sample consists of 5,639 students who were rising into fourth grade in the summer of 2013. We are examining the extent to which participating in these programs affects math, reading, and social-emotional outcomes. We will follow the students through 2016, reporting on outcomes after one year and then two years of summer program participation. Throughout the course of this study, we will examine performance on tests we administer, social/emotional surveys we administer, and state assessments, as well as grades, grade retention, and student behavior during the school years. In this AERA session, we will describe fall math, reading, and social/emotional outcomes for these students after participating in one five- or six-week summer program.

The logic model undergirding these programs is that by offering summer programs that provide academic instruction, participating students will not lose knowledge and skills during the summer, and they may even advance. Across our sample, over 85 percent of the students qualify for free or reduced price meals. Prior research indicates that these students lose knowledge and skills over the summer in both math and reading.

This is a mixed-methods study. We employ a randomized control trial design, with qualitative data on implementation gathered through classroom observations, interviews, and surveys. Measured outcomes include student scores in the fall on Pearson’s GRADE (reading) and GMADE (math) assessments, administered across the districts specifically for this study. We also administered the Devereoux Student Strengths Assessment-RAND Research Edition to the fall homeroom teachers of these students. Based on these measures, we will present intent to treat outcomes, as well as treatment on the treated outcomes given the 20 percent no-show rate observed across the sample. We will also present the result of non-experimental models that examine the extent to which variables such as student attendance, hours of academic instruction, and the quality of instruction, mediate or moderate the academic and social/emotional outcomes.

The study results are embargoed until the winter. All of the data have been analyzed, however, and we look forward to presenting the results at AERA. Understanding the potential for district-run, voluntary summer learning programs serving large numbers of public school students to stem summer learning loss is an important endeavor for educators, researchers, and policymakers.

Authors

• Catherine H. Augustine, RAND Corporation

• Jennifer Sloan McCombs, RAND Corporation

• John Pane, RAND Corporation

• Heather L. Schwartz, RAND Corporation