Comparing Classroom Video Analysis and Other Measures of Teacher Knowledge in the Evaluating Developing Mathematical Ideas Impact Study

• In Event: Further Exploration of the Classroom Video Analysis Approach for Measuring Usable Teaching Knowledge in Mathematics

Abstract

Evaluating DMI is an NSF-funded (DRL-1019769) experimental study of the impact on teacher knowledge, teaching practice, and student achievement of elementary teacher participation in the innovative DMI math professional development program (Schifter, Bastable, & Russell, 1999-2008). The logic of the intervention is that by deepening teachers’ understanding of mathematical ideas and how students think and learn about those ideas, teachers will change their teaching practice, especially to emphasize richer mathematical discourse, which will improve student understanding and ability to justify mathematical ideas.
Grades 1-5 teachers from eight districts in Massachusetts and North Carolina volunteered to be randomly assigned to take Building a System of Tens (BST) and Making Meaning for Operations (MMO) intensive math PD in the first year of their participation, or to wait a year and serve as the comparison group. Several assessments of teacher knowledge, videotaped classroom practice, and student achievement are being collected and analyzed from all teachers across two years. Analyses look for impacts in each area, and links among them.
Teacher knowledge measures include the Math Knowledge for Teaching (MKT) measure (Ball, Hill, Rowan, & Schilling, 2002; Hill, Schilling, & Ball, 2004); an open-ended assessment aligned with DMI content (OE) (Higgins, Bell, Wilson, Oh, & McCoach, 2007); and a new Classroom Video Analysis (CVA) measure (Kersting, 2008) focusing on number and operations concepts. Since the link between changes in teacher knowledge and changes in teaching practice is key to assessing the impact of DMI, a measure of usable knowledge like the CVA was important for the study. To match the topic areas of the DMI seminars we collaborated with Kersting to develop a new CVA scale, yielding 11 video clips on numbers and operations, which produced reliable scores.
Initial high correlations among the MKT, the OE measure, and the new CVA suggest they are measuring similar, but not the same kind of knowledge. The MKT and OE show statistically significant differences for teachers who completed DMI, after controlling for pre-DMI MKT scores, suggesting that DMI teachers increased their mathematics knowledge. Though we did not find significant treatment differences for the CVA or any of its subscales for the subsample of teachers who have so far completed the CVA, we did observe DMI teachers more deeply analyzing student thinking in the video clips and demonstrating an overall increased depth of interpretation compared to teachers who did not participate in DMI seminars, which is consistent with DMI objectives.
One possible interpretation is that although teachers in the DMI program clearly gained relevant knowledge as measured by the MKT and OE, not all of that knowledge is yet usable, reflected in the smaller effects observed for the CVA. At the symposium we will further explore this interpretation by examining the statistical results in light of actual teacher responses on the CVA instrument and by linking teacher knowledge scores to measures of instructional practice and student learning. We will report on the complete sample for Year 1 and on a subsample that compares Year 2 to Year 1.

Author

• James K.L. Hammerman, TERC