Measuring the Efficacy of Research-Based Revisions to Middle School Mathematics Curricula

• In Event: Bridging Research and Practice: Findings From the National Center on Cognition and Mathematics Instruction

Abstract

Research on student learning has yielded cognitive principles that have the potential to inform the design of materials and instructional practice. The Institute for Education Sciences summarized some of the most promising research findings and formed recommendations in the IES Practice Guide: Organizing Instruction and Study to Improve Student Learning (Pashler, 2007). To establish whether applying these recommendations to classrooms would improve student outcomes the National Center on Cognition and Instruction brought together expertise in cognition, instruction, assessment, research design and measurement, mathematics education, and teacher professional development to redesign components of a widely-used middle school mathematics curriculum, the Connected Mathematics Project (CMP). The entire year of seventh grade CMP materials were revised by applying four cognitive principles of learning from the IES Practice Guide: (1) integrating visual and verbal information, (2) interleaving worked examples and self-explanation with problem solving (3) spacing practice, and (4) using formative assessment.

We report findings from 108 seventh grade teachers at 73 schools. Teachers were randomly assigned at the school level to either teach using the existing CMP2 curricula or to teach using the revised CMP2 materials. Over an entire school year, we collected data including demographic information, student performance on the Mathematics Diagnostic Testing Project (MDTP), unit posttest results, and teacher scores on the Mathematical Knowledge for Teaching (MKT) measure.

To determine whether the revised materials were associated with improved student outcomes, we conducted separate linear mixed effects ANCOVA models with additional random effect terms to account for the nesting of subjects within teachers and schools on the MDTP and seven unit posttests. These models included fixed effect covariate interactions with the treatment status at the student (e.g., MDTP pre-test, ELL status), teacher (e.g., MKT score, fidelity of implementation) and school levels (e.g., proportion FRL status at the school, proportion students proficient on the state mathematics exam). Results indicated that estimates for the treatment status were in the predicted direction for all 8 models (higher overall performance in the treatment condition relative to the control; Hedge's g ranged from 0.18 - 0.34). There was a significant effect of treatment status at alpha < .10 for two of the models and a significant effect of treatment status at alpha < .05 for one of the models. In addition, several covariates consistently moderated the effect of treatment status. In four of the models the effect of the treatment was enhanced for students with teachers who had lower scores on MKT and fidelity measures (p < .05; however, all models showed this same trend). The effect of the treatment was also enhanced in students with lower pre-test scores for 4 of the models at the alpha = .10 level (but all models showed the trend). Overall, these results suggest that incorporating cognitive principles with classroom instruction and practice can support students' mathematics learning and may help to close achievement gaps in student subpopulations.

Authors

• Jodi Davenport, WestEd

• Yvonne Kao, WestEd

• Bryan Matlen, WestEd

• Perman Gochyyev, University of California - Berkeley

• Steven Arnold Schneider, WestEd