Poster #38 - Comparing Off-line and On-line Measures of Metacognition in 8th Grade Social Studies

• In Event: 1-005 - Poster Session 01
  In Poster Session: PS 01 Section - Cognitive Processes

Integrative Statement

Off-line methods of assessing metacognition are collected either before or after task performance, whereas on-line methods assess metacognition during task performance. The assessment methods each have advantages and disadvantages (Veenman, Van Hout-Wolters, & Afflerbach, 2006). The present study compared off-line and on-line measures of metacognition, based on a conceptual model of metacognition proposed by Ambrose, Bridges, DiPietro, Lovett, and Norman (2010). As a critical element of self-regulated learning, metacognition comprises several components – assessing task demands, evaluating personal strengths and weaknesses, planning, applying strategies to improve learning, monitoring performance, and reflecting and adjusting to feedback as needed. The goals of the study were to determine: (1) the degree of convergence between the off-line and on-line measures; (2) the degree of criterion validity shown by the off-line and on-line measures; and (3) whether increased experience with the task which was the focus of the on-line assessment improved quality of measurement. Participants were 8th grade students (N = 20) enrolled in a United States history course, which included a unit on westward expansion. The on-line assessment required students to play The Oregon Trail; when students encountered an obstacle, they were prompted to reflect on their response to the obstacle, which was then coded for degree of metacognition evidenced. The on-line assessment occurred once when students were “novices” and then again when they were “experts” (having played six additional times on their own). The off-line assessment of metacognition consisted of a 35-item self-report scale (Cronbach’s  = .89). The items in the scale measured how frequently students had engaged in specific metacognitive activities in the social studies course over the past month. Correlations between the off-line and on-line measures of metacognition were weak, both when the on-line assessment involved “novices” (r = .33) and “experts” (r = .13). On-line measures of metacognition were nominally better predictors of success playing The Oregon Trail, compared to the off-line measure, but the differences between the correlations were not always statistically significant (see Table 1). On-line measures of metacognition were better than the off-line measure at predicting students’ course grades (see Table 1). Mean levels of metacognition did not differ at the “novice” and “expert” assessments, although “experts” traveled significantly farther on The Oregon Trail but had significantly fewer survivors (see Table 2). This research with a multi-method design adds to the literature on assessment of metacognition. The on-line measure of metacognition showed a promising level of criterion validity when students were assessed after gaining some expertise with the task.

Authors

• Michelle Treglia, Department of Psychology, Trinity College
  Non-Presenting Author

• David A. Reuman, Trinity College
  Presenting Author

• Dina Anselmi, Trinity College
  Non-Presenting Author