Assessing Effects of Learning Mathematics Through Representations on Student Learning

• In Event: Learning Mathematics Through Representations: A Research-Based Curriculum

Abstract

[Figures in Session Summary]

We report selected findings from an efficacy study designed to contrast learning gains in 11 LMR and 10 comparison classrooms (N=522 students), with groups matched for teacher background and student demographics. Students were assessed on paper-pencil measures four times in relation to the intervention: pretest, interim, posttest, and year-end tests. These tests contained integer and fraction knowledge items adapted from several sources, including participants’ adopted curriculum (Everyday Mathematics), LMR, NAEP, and California’s testing program; the approximately 30 item assessments contained 18 recurring items that enabled item response theory (IRT) scaling. The analyses reported focus on the first three assessments. (Data from the 4th assessment are currently being processed, and a fifth assessment – state test scores – will be released Fall 2011.)

1. Intervention Effects. Assessment data were analyzed using a three-level IRT model (Adams, Wilson, & Wu, 1997; Kamata, 2001; Kamata & Cheong, 2007), which included indicator covariates for test, assignment group, and their interaction. Figure 1 shows the distribution of estimated ability from the multilevel IRT model over test. At pretest, there were no group differences (z=0.65; p=0.513); by posttest, the estimated ability for LMR students was 1.7 logits higher (s.e.=0.10), than for comparison students (z=17.29; p < 0.001) with a large effect size of 1.2. The LMR distribution at post-test is negatively skewed, indicating a larger than average growth for many students.

2. Patterns of LMR gains by classroom. Figure 2 displays classroom distributions for LMR pre, interim, and posttest scores. Two patterns of gains emerged -- classrooms in which distributions (a) maintained their shape as performances improved vs. (b) became negatively skewed. We are investigating relationships between these patterns and differences in lesson implementation through analysis of lesson videos and teachers’ lesson logs.

3. Properties of learning in LMR classrooms. We used a two-level linear model (Raudenbush & Bryk, 2002) that was fit using proportion correct scores on two subsets of the recurring problems, those with and without number line representations (outcome variable). Figure 3 shows the mean proportion of correct responses for the two item subsets. LMR students made gains on both number line (0.24, s.e. =0.01, p < 0.001) and non-number line problems (0.34, s.e. = 0.01, p < 0.001), showing greater average gains than comparison students on both item types (NL:0.26, s.e. = 0.04, p < 0.001; non-NL:0.22, s.e.=0.02, p < 0.001). LMR students did not show greater differential growth between the two problem types than comparison students (z=1.64, p = 0.101), suggesting that LMR students used LMR number-line instruction to generate solutions to non-number line tasks.

Significance. The analyses completed to date provide initial evidence of the efficacy of LMR. If the symposium is accepted, we will present additional analyses of LMR effects.

Authors

• Ronli Diakow, University of California - Berkeley

• Geoffrey B. Saxe, University of California - Berkeley