A Quasi-Experimental Study of Next Generation Science Standards Curriculum and Online Professional Development on Three-Dimensional Learning Outcomes

• In Event: Innovations in Video-Intensive STEM Professional Development in Online and Blended Environments

Abstract

Objective
A Framework for K-12 Science Education (NRC, 2012) and the Next Generation Science Standards (2013) call for new curriculum materials to support student engagement with three-dimensional learning, PD experiences to support teachers in using the materials, and assessments to measure students’ mastery of these ambitious standards. To date, there has been little or no research into the efficacy of curricula designed for the NGSS, supported by online video-based PD, and assessed using instruments designed to measure students’ three-dimension learning outcomes. The project and research we describe begins to fill that void.

We describe a four-year project that:
• developed an NGSS digital middle school body systems unit;
• developed an NGSS assessment;
• developed an 11-week online video-based PD model to support curriculum enactment;
• conducted a quasi-experiment of the intervention on student three-dimensional learning outcomes; and
• measured pre/post changes in teacher knowledge and practice.

Perspective/Theoretical Framework
We provided teachers with digital curriculum materials that attended to NGSS design criteria and that are expected to promote three-dimensional learning (NRC, 2012). We also provided PD in an online environment built on a video-based PD model that incorporated features that have been predicted to enhance the efficacy of PD (Desimone, 2009). The video-based analysis-of-practice model was adapted from a highly successful face-to-face model (Roth et al., 2011; Authors, 2017). Our online version allowed us to reach a national audience of teachers during the field test.

Methods
We conducted a quasi-experiment to investigate the efficacy of the intervention in supporting students’ three-dimensional learning outcomes. In this two-year cohort control study, teachers used their extant materials the first year, participated in online PD in the summer between school years and into the fall of the second year, and used the intervention materials in the fall of the second year. Comparison students were taught using extant materials, and
treatment students used intervention materials. The design allowed us to control for individual teacher effects. In intervention research, teacher effects can contribute as much as 21% of the variance in student outcomes (Nye, Hedges, & Konstantopoulos, 2004).

Data
The student assessment included multiple choice and open-ended response items. Teachers completed a content pretest and posttest and filmed themselves teaching a lesson before and after the summer PD. We analyzed student data using three-level HLM, with students nested within classes, and classes nested within teacher. We analyzed teacher data using OLS regression with teacher pretest and years of experience as explanatory variables.

Results
Teachers demonstrated significant increases in content knowledge (p < .001, d = .88) and classroom instructional practice (p < .001, d = 1.19). Teacher posttest scores did not depend on teacher years of experience. Treatment students modestly outperformed comparison students. The difference was statistically significant (p = .007; d = .280). See Tables 1-3.

Significance
The findings support the thesis set forth by the NRC (2012): when teachers use curriculum designed for the NGSS and are supported by extensive PD in an online environment, students perform better on assessments designed to measure ambitious three-dimensional learning goals.

Authors

• Betty Stennett, BSCS Science Learning

• Susan M. Kowalski, BSCS Science Learning

• Lindsey Mohan, BSCS Science Learning

• Jeffrey Snowden, BSCS Science Learning

• Lisa Carey, BSCS Science Learning

• Heather Young, Oregon Public Broadcasting

• Catherine Stimac, Oregon Public Broadcasting