How Does Student Learning in Multiple Literacies in Project-Based Learning (ML-PBL) Compare to Student Learning in "Business-as-Usual" Classrooms?

• In Event: Elementary Students' Academic and Social Emotional Learning Through Project-Based Learning: Results of an Efficacy Study

Abstract

Objective
This study describes the results of a cluster-randomized efficacy trial of the Multiple-Literacies in Project-Based Learning (ML-PBL) program for 3rd-grade science. ML-PBL is a system for science learning that includes 1) professional development on NGSS, project-based learning (PBL; Author, 1991) and specifics of the curriculum; and 2) four, nine-week science units for 3rd grade aligned with NGSS and Common Core standards, designed using PBL principles, and emphasizing math and literacy.

Theoretical Background
PBL emphasizes students’ understanding of phenomena by engaging deeply with a driving question to develop explanations and models of the phenomena. Students develop usable understanding of science ideas to meet the learning goals elucidated in the NGSS. Previous studies of PBL in science show that it is a promising approach for improving student learning (e.g., Author, 2015).

Method
51 elementary schools were randomly assigned to treatment or control conditions. The treatment consisted of the aforementioned program, while the control condition received professional development on the NGSS only, and continued with “business-as-usual”. This study compares the treatment and control conditions to determine the causal effect of the PBL treatment compared to the control condition.

Table 1 presents descriptive statistics of the treatment and control samples. Meeting the What Works Clearinghouse guidelines (IES, 2018), we obtained independent pre-and post-test summative scores. Pre-test benchmark scores in reading and mathematics are being obtained for all sampled students. An independent post-test developed by the Michigan Department of Education was given to all sampled students.

We are analyzing the data using a hierarchical linear model (HLM; Raudenbush & Bryk, 2002) to assess the difference between the treatment and control condition on science achievement. HLM is preferable to other methods as it nests students within schools to account for clustering that occurs as a result of assignment of schools to treatment and control conditions rather than assigning individual students (Bloom, 2005). We estimate the following model:

PostTestScoreij = β0 + β1Treatmentj + β2PreTestij + β3MeanPretestj + eij + uj

where PostTestScoreij is the summative post-test score for student i in school j. β1 is the coefficient of interest, indicating the mean difference between the treatment and control groups on the post-test. We control for individual pre-test (PreTestij) and school-mean pre-test scores (MeanPreTestj) and include error terms at the student (eij) and school (uj) levels.

Results and Significance
Data for this study continues to arrive, is entered into our data files and analyzed. To give an update on our data: (1) for the pre-test objective covariate, schools using the NWEA reading, language usage and math tests, the mean average for the sampled treatment students is 199.02 and 198.13 for controls which when converted is at the state average for beginning 3rd graders. (2) for the post-test, the average score for the treatment students in these schools is 0.33 and 0.32 for the controls which are significantly different for this preliminary sample.

The results of this study show promise of the value of PBL to support student learning in the elementary grades.

Authors

• I-Chien Chen, Michigan State University

• Quinton Baker, Michigan State University

• Christopher R Klager, Michigan State University

• Barbara Schneider, Michigan State University