The Unique Challenges of Conducting Evaluations of Out-of-School-Time Programs: Lessons Learned From the Field

Session Type: Symposium

Abstract

Researchers with experience implementing evaluations in the school-day space may be tempted to approach studies in OST with similar expectations for potential issues that may arise. However, as this panel will demonstrate, there are unique challenges to designing and implementing evaluations in this space. Researchers from ExpandED Schools, EDC, and IMPAQ International will draw from their experiences to highlight hurdles they’ve encountered and solutions they’ve engineered when conducing robust OST evaluations. The discussion will be structured around four categories of methodological challenges (and solutions) unique to OST: recruitment and retention of schools and students; fidelity to program models and dosage; issues of data collection and consent; and selecting metrics to assess impact that balance funder needs with practitioner interests.

Sub Unit

• SIG-Out-of-School Time

Chair

• Cheri L. Fancsali, Research Alliance for New York City Schools

Papers

• Recruitment, Retention, and Partnership Challenges in Rigorous Out-of-School-Time Evaluation Designs - Andrea D. Beesley, IMPAQ International

• Issues With Fidelity of Implementation of Educational Interventions in Out-of-School Time - Kathryn Brohawn, ExpandED Schools, Inc.; Lizzie Murchison, ExpandED Schools

• Data Collection Challenges to Consider When Designing and Implementing Out-of-School Time Evaluations - Carrie S Liston, Education Development Center, Inc.

• Selecting Metrics to Assess the Impact of Out-of-School-Time Programs: Balancing Practitioner Goals With Funder Expectations - Sophia Mansori, Education Development Center, Inc.; Erin Stafford, Education Development Center, Inc.

Discussant

• Leslie Karn Goodyear, Education Development Center, Inc.