What Makes Reading Difficult? Contributions of Passage, Task, and Reader Characteristics on Item Difficulty

• In Event: Text Comprehension: What We Know About the Dance Between Reader, Text, and Task in Reading Comprehension

Abstract

Reading comprehension (RC) is a multi-faceted, multi-layered construct (Broek & Espin, 2012; Duke, 2005; Graesser & McNamara, 2011; Perfetti & Stafura, 2014), which manifests through complex interactions among three broad factors: the reader, the passage, and the task, all residing in a particular socio-cultural context (RAND Reading Study Group, 2002). Drawing on the item difficulty modeling paradigm, this study examined how these three factors as well as their interactions affected comprehension difficulty. Specifically, the study used explanatory item response models (De Boeck & Wilson, 2004) to analyze item response data from an a vertically-scaled online assessment, which included a wide range of readers (n=10,547) as well as of passages (n=48), covering grades 1 through 12+. Analyses indicated that text features, as measured by computational text analyzers (i.e., Lexile, Coh-Metrix, and TextEvaluator), explained over half the variance in item difficulty, and more than task features as coded mostly by human raters (after controlling for student general vocabulary knowledge).
Specifically, sentence length, word frequency, syntactic simplicity, and temporality (i.e., the extent to which the text has time markers) were found to significantly affect comprehension difficulty in both model building and cross validation analyses. Further, small but significant interaction effects were found, indicating that these textual effects were moderated by student general vocabulary knowledge as well as task demands as captured by item types. In general, readers with higher vocabulary knowledge benefitted more from traditional textual affordances (e.g., shorter sentences, familiar words) than their peers with lower vocabulary knowledge, especially when questions asked them to recall specific localized information without accessing the source passage. However, a reverse effect was found with temporality: readers with lower vocabularies benefited relatively more from passages with more time markers, while readers with higher vocabularies benefited relatively more from passages with few temporality. The implications of these findings as well as their limitations are discussed as they relate to the measurement of RC and to instructional practice.

Authors

• Mark R. Wilson, University of California - Berkeley

• Yukie Toyama, University of California - Berkeley