Making Thinking Visible Through Multimedia Artifacts

• In Event: The Video Mosaic Collaborative: An Online Professional Development Resource for Mathematics Education and the Learning Sciences

Abstract

Objectives

Our research investigates using multimedia artifacts for assessing knowledge about the development of mathematical reasoning. Guiding research questions include:
1. To what extent can we use a cyber-enabled multimedia construction tool to assess how well learners justify their arguments about children’s reasoning?
2. How does variation in course context and tasks relate to differences in multimedia artifacts produced?
Theoretical framework
A view of learners as active constructors of personally meaningful mathematics guided decades of research on how students develop mathematical ideas and reasoning (Baroody & Ginsburg, 1990; Maher & Davis, 1990), producing a unique video collection from longitudinal and cross-sectional studies (Agnew et al 2010; Palius & Maher, 2013). Part of the collection is housed in a repository, the Video Mosaic Collaborative (VMC, www.videomosaic.org). The VMC has a cyber-enabled tool, VMCanalytic (Figure 1), for creating multimedia artifacts. Our current research engages teachers in generative activity to construct multimedia artifacts for sharing what they have learned about the development of mathematical reasoning through studying VMC videos. Teachers’ work with VMCAnalytic utilizing a video editing tool. Similar to tools such as WebDiver (Zahn et al., 2010), the VMCAnalytic enables selection of video segments that can be annotated and remixed to form multimedia narratives for a variety of purposes (Hmelo-Silver et al., 2013). Once shared, VMCAnalytics become objects for discussion and refinement.


Learning technologies provide new opportunities to teach and assess complex skills (Derry et al., 2006; Pellegrino & Quellmalz, 2010). Evidence for assessment can be found in artifacts that learners create with technology (de Jong et al., 2012). The VMCAnalytic provides opportunities for learners to make their thinking visible and thus provides opportunities for assessment. The VMCAnalytic can elicit complex performance that allows instructors to monitor students’ developing understanding.

Methods and Data Sources
Data include 63 VMCAnalytics that were created over the last two years by participants in seven graduate courses in mathematics education or research methods (Table 1). For each category in Table 1, the VMCAnalytics were rated on a 0-3 scale. Two coders scored the sixty-three analytics with inter-rater reliability of 88.72%.

Results
Table 1 contains descriptive statistics for the seven classes. The Wilcoxon signed-rank test found significant differences (all p < 0.05) between DBR Fall 2011 and all the other classes except Critical Thinking and Reasoning for the categories of overall description, the clips connecting meaningfully, claims are backed, overall clarity and coherence, and event relevance. We also found significant differences in event relevance between Introduction to Mathematics Education in Spring 2012 and all other classes. These patterns of discrimination in relation to course content and level provide evidence of construct and face validity.

Significance
The VMCAnalytic shows promise of being a useful tool for formative and summative assessment. It makes students’ thinking visible and open for discussion and revision. Students’ evolving understanding becomes transparent, and instructors can see their intellectual journey in thinking critically about children’s mathematical thinking.

Authors

• Cindy E. Hmelo-Silver, Indiana University

• Carolyn Alexander Maher, Rutgers University

• Marjory Fan Palius, Rutgers University

• Robert Sigley, Rutgers University - New Brunswick/Piscataway

• Alice S. Alston, Rutgers University