Think-Alouds and Recalls and Validating Measures of the Multiple-Choice Online Causal Comprehension Assessment (MOCCA)–College

• In Event: First-Year Results From the College Version of the Multiple-Choice Online Causal Comprehension Assessment (MOCCA-College)

Abstract

Objective. The MOCCA-C (multiple-choice online causal comprehension assessment-college) is a measure our research team is developing and refining to assess reading comprehension for college students. The purpose of this study is to assess the validity of the MOCCA-C through correlations with other reading measures.
Theoretical Framework. According to the construction-integration model of text comprehension, the reader needs to make connections with the ideas in the text in a meaningful manner in order to comprehend the text (Kintsch, 1998). Making these connections (known as inferences) is necessary for readers to understand the causes and effects of events in the next (causal coherence; van den Broek, 1994).
Methods and Data Source. The MOCCA-C consists of 50 seven-sentence items with the sixth sentence blank. Readers need to choose which of three possible sentences best completes the text with one option being the causally coherent inference necessary for the text as a whole to be comprehensible (no time limit). Sixty-three undergraduate students who took the MOCCA-C online assessment participated in face to face data collection on validating reading tasks. Participants read and articulated their thoughts while reading aloud after each sentence (i.e., a think aloud) four texts, then orally recalled what they remembered from the text after reading each text. They also completed the TOWRE-2 sight word and decoding measures (Torgesen et al., 2012) and the Nelson-Denny Subtest of Reading Comprehension (time limited tasks; Fishco, 2019).
The think aloud responses were parsed into idea units (noun and verb combinations that represent an idea) and coded for a range of responses including inferences. Because the MOCCA-C is intended to assess causal coherence skills, we categorized whether the idea units were highly connected in the cause-and-effect structure of the text or not. The recall was also parsed and coded for the idea unit it best represented.
Results and Significance. Two metrics for MOCCA-C performance were examined: number correct (accuracy) and seconds per correct item (efficiency).
Overall recall was positively correlated with MOCCA-C accuracy, but this correlation was constrained to idea units that were not highly connected. In general, highly connected ideas are more likely to be recalled than other ideas in the text (van den Broek, 1994). Moreover, readers in college typically have sufficient skill to be sensitive to the structural centrality of the text, meaning they would recall highly connected idea units (van den Broek & Helder, 2017). Therefore, it is possible that readers across causal coherence skills levels recalled the highly-connected units, but remembering other idea units required more skill. However, efficiency and decoding correlated with highly connected idea units as did the number of inferences, indicating making connections may be linked with reading fluency in this population. The sight word and decoding scores were negatively correlated with the efficiency measure, but not with overall accuracy. This is likely because more fluent readers who were able process the text more quickly were also able to identify or decode individual words more quickly.

Authors

• Virginia Clinton-Lisell, University of North Dakota

• Benjamin Seipel, CSU, Chico

• Sarah Elizabeth Carlson, Georgia State University

• Mark L. Davison, University of Minnesota

• Terrill Taylor, University of North Dakota

• Surjya Bajpayee, North Dakota State University

• Heather Ness, Georgia State University