Evaluating the Impact of New York City's Expanded Success Initiative on Students' College Readiness

• In Event: 63.043 - Improving College-Going for Black and Latino Young Men: Evidence From New York City's Expanded Success Initiative

Abstract

Objective
The purpose of this study is to determine whether students who are exposed to ESI-related interventions and supports achieve better college readiness outcomes than they would have if their schools had not been involved in ESI. This impact study will measure both academic outcomes and non-academic outcomes related to college readiness.

Theoretical Framework
The theory of action driving ESI is that an integration of three different school-based domains—academics, youth development, and school culture—will improve college readiness for young men of color. In terms of academics, ESI is based on research which shows that rigorous high school coursework, four years of math and science, and opportunities to enroll in college-level courses help students remain on track for college success (Adelman, 1999; Scramm, 2008). Related to youth development, ESI encourages schools to implement programs that strengthen student relationships with adults and incorporate restorative approaches to school discipline that reduce suspension (Bryk & Schneider, 2002; Hughes, Cavell, & Wilson, 2001; Skirba, et al. 2002). The school culture domain is based on strategies that promote a college and career focus through college-oriented programs and authentic workplace experiences (Conley, 2007; Riegle-Crumb, 2010). Our study will assess the degree to which students achieve growth on measurable outcomes related to each of these domains.

Methods and data sources
This study utilizes comparative interrupted time series (CITS) to compare the outcomes for students currently enrolled in ESI schools with those of 1) previous cohorts of students in the same schools prior to ESI and 2) students in a set of comparison schools. Data sources include administrative records from the New York City Department of Education on attendance, credit accumulation, and course grades. The analysis of ESI’s impact on key non-academic outcomes will rely on survey data collected specifically for the ESI evaluation. Surveys were administered to 9th and 10th grade students in all 40 ESI schools and a group of 23 comparison schools. We will calculate mean scores for individual students, school-level scores, and differences in responses within schools and between schools.

Results
Our results will show 1) differences, if any, in outcomes between ESI students and previous cohorts in the same schools, 2) the level of variation across ESI schools, and 3) the difference, if any, between ESI schools and comparison schools. In addition, we hope to draw associations between variations in impact and differences in implementation. For example, did students in schools with higher implementation fidelity or intensity also achieve greater academic growth? This impact study will not only evaluate the effectiveness of ESI, but also help highlight key differences between schools that could help improve the initiative in future years.

Scholarly significance
Many race- and gender-based initiatives such as ESI have emerged in recent years. But there has been less empirical research on whether, or to what degree, these types of initiatives are effective or actually make a difference for students. By including non-academic outcomes measured by a student survey, this study also contributes to our understanding of student success beyond academic outcomes like test scores.

Authors

• Adriana Villavicencio, New York University

• Sarah L Klevan, Learning Policy Institute