Impacts of a K-2nd Virtual Science Program: A Randomized Controlled Trial During the COVID-19 Pandemic

• In Event: How Can Children Optimize Their Online Learning Experiences With Educational Technology?

Abstract

Introduction
For decades, American schools, particularly those serving economically disadvantaged students, have struggled to provide young students with science education. Furthermore, school closures in response to the COVID-19 pandemic, have accentuated gaps in access to science education. While educators pivot to deliver distance instruction, it is important for us to identify feasible remote learning strategies for science content, to minimize the gap in science learning gains.

[PROGRAM] is designed to provide opportunities for early science engagement and learning across diverse student populations through transmedia playlists that are accessed via a web-browser. The program embeds K-2nd grade Next Generation Science Standards (NGSS) across all activities, including a new television show, digital games, non-fiction e-books, hands-on activities, and a digital notebook.

Hypothesis
The present randomized controlled study explored whether [PROGRAM] could support students’ science engagement and learning in the context of responding to the COVID-19 pandemic. We predicted that students who use [PROGRAM] would be more engaged in science and have more increases in science learning than students who do not.

Study Population
Thirty-four second-grade teachers (n=793 students) from 20 California schools, who serve
economically disadvantaged students were randomly assigned to a treatment (n=17 teachers) or control group (n=17 teachers).

Methods
The study began with in-person school implementation and transitioned to distance learning due to the COVID-19 pandemic. Teachers were randomly assigned to either use four transmedia playlists (treatment) or conduct their business-as-usual science lessons (control). Treatment teachers implemented [PROGRAM] two to three days per week for 50 minutes per day and finished each playlist in two weeks. The study relied on hierarchical linear model techniques to determine whether [PROGRAM] was more effective than control for improving science learning as measured by an online NGSS aligned science assessment. Qualitative and quantitative data collected from weekly teacher logs, virtual classroom observations, teacher interviews, playlist telemetry, and parent surveys were used to compare program implementation and student engagement between the conditions.

Results
Student outcome data were collected from 319 students in 24 classrooms, representing 62% of students who had distance learning accessibility. The baseline equivalency tests indicated no difference between the treatment and control groups. The primary results indicated that [PROGRAM] was positively associated with gains in students’ knowledge in science (effect size = .15). Treatment teachers reported that [PROGRAM] helped students with the transition to distance learning and that students became more comfortable with technology throughout the implementation. Students were using more scientific vocabulary and making connections between the activities in [PROGRAM] and their own lives during class discussions and in written work.

Authors

• Momo Hayakawa, Twin Cities PBS
  Presenting Author

• Linlin Li, WestEd
  Non-Presenting Author

• Joan Freese, Twin Cities PBS
  Non-Presenting Author