A Randomized Controlled Trial of an After-School Science Program for Middle-Grade Students

• In Event: Better Together: Opportunities, Challenges, and Outcomes of Formal and Informal Educator Collaborations

Abstract

Objectives and theoretical framework

Design2Learn, funded through the US DOE’s Investing in Innovation fund, is an afterschool program for middle school students—a period in which performance in science often declines sharply. The program is aimed at increasing students’ interest, engagement, and academic performance in science. Students participate in weekly science instruction after school, facilitated by a three-person educator team (one certified teacher; two informal educators) and designed to connect science learning goals to real-world activities. Students enroll in Design2Learn in sixth grade, and continue the program through eighth grade.

The program combines three strategies:
1) collaborative teaching, which brings together certified teachers and staff employed by nonprofit organizations that provide school-based afterschool programs to jointly deliver instruction;
2) curricular bridging, which coordinates in-school and afterschool instruction to create seamless student learning experiences; and
3) design-based learning, which provides students with hands-on, real-world activities.
This paper will describe the D2L model for teacher collaboration and the results of the first year of an implementation and impact study of the model. The presenter will also describe implications of the study results for policy and practice.


Methods

This study is a randomized controlled trial with 32 afterschool programs located in NYC public schools. Programs were randomly assigned to participate in D2L (treatment) or business as usual STEM programming (control). Teacher teams in the treatment sites participate in the D2L professional development and coaching support. The confirmatory impact research question addressed by this study is: What is the impact of the opportunity to participate in Design2Learn for at least two years on 8th grade students’ interest and engagement in science? The implementation study alsos explores program fidelity of implementation and implementation facilitators and barriers.


Data Sources

From students, we collected pre/post interest and engagement surveys, attendance (in the afterschool and in school), and science achievement (standardized tests and course grades) data. From educators, we collected end of the year surveys to measure fidelity of implementation, changes in practice, and successes and barriers to implementation. We also conducted observations of treatment and control program activities and interviewed program staff to describe what D2L looks like ‘on the ground’.


Results

Findings from the year 1 educator survey suggest promising results from the teacher collaboration. Selected results include:
Only 16% reported finding time to collaborate with co-teachers was challenging. None reported that the working relationship with co-teachers was challenging.

-36% agreed/strongly agreed that D2L helped youth understand science concepts better

-Over 80% reported D2L increased student knowledge and interest in science to a moderate or great extent.

-72% reported D2L activities supported students' in school science learning.

-92% agreed/strongly agreed they are open to receiving feedback about activities from their collaborators, 88% agreed/strongly they respect their teaching partners and feel respected by them, and 88% agree/strongly agreed that collaborating enhanced their ability to lead successful science activities and improved the quality of the activities offered.

We will also present results of the first year impacts on student interest and engagement.

Authors

• Cheri L. Fancsali, Research Alliance for New York City Schools

• Ben Schwab, Research Alliance for New York City Schools

• Paulina Toro Isaza, Research Alliance for New York City Schools

• Emma Banay, Expanded Schools