Square Peg, Round Hole: Negotiating the Appropriateness of "Gold Standard" Research Design for Out-of-School Time

Session Type: Roundtable Session

Abstract

The out-of-school time (OST) space presents unique circumstances to consider when designing research studies. Typical “gold standard” approaches, such as randomized controlled trials (RCTs), are not always a feasible method of studying OST program effectiveness. Beyond challenges related to funding and capacity, rigid design requirements of RCTs may not offer a true assessment of flexible OST contexts. Furthermore, equity considerations around access and control conditions are paramount. This symposium examines three research designs: a completed RCT, an in-progress quasi-experimental design that adapted lessons learned from the RCT, and a flexible mixed-methods approach. The panel includes both researcher and practitioner perspectives with pairs of evaluators and program leaders; pros and cons of different methodologies and future research considerations will be discussed.

Sub Unit

• SIG-Out-of-School Time

Chair

• Isabella Fante, ExpandED Schools

Papers

• Lessons Learned: Design Challenges and Equity Considerations of a Randomized Controlled Trial in After-School - Emma Banay, Overdeck Family Foundation; Fran Agnone, Sanitation Foundation; Cheri Fancsali, Research Alliance for New York City Schools

• Adapting Lessons Learned: A Randomized Controlled Trial–Turned–Quasi-Experimental Design That Developed Buy-In and Adapted to Local Context - Candace Brazier-Thurman, ExpandED Schools; James J. Kemple, New York University

• A Flexible Mixed-Methods Approach to Evaluation in After-School: Benefits and Considerations - Joel Nunez, ExpandED Schools; Isabella Fante, ExpandED Schools

Discussant

• Katie Brohawn