A Proposal for New Comprehension Assessments: Addressing Issues of Text Complexity and Reading Process

• In Event: Text Complexity: What Is It, How Is It Measured, and, After All, Why Is It Important?

Abstract

Purpose
The purpose of this paper is to build upon theories and research of text complexity and reading comprehension to propose a new model of comprehension assessment. This model should enable reading researchers to describe particular interactions of reader, text, and context that contribute to text complexity and, as a result, reading difficulty.

Perspective
We take as a starting place for our model of comprehension assessment, a process used by Kirsch (2001, 2003) to develop the National Assessment of Adult Literacy. The first four of six steps in the process are central to issues related to text complexity: 1) defining reading comprehension, 2) organizing the domain, 3) specifying task characteristics, and 4) identifying and operationalizing variables. In addition, we draw on seminal work in cognitive, curricular, and pedagogical research (see Method) that bear on conceptions of reading comprehension and text complexity.

Method and Data Sources
We began with a review of research on reading comprehension and text complexity. These included: the interactive and contextual nature of reading comprehension (e.g. RAND Reading Study Group, 2002); cognitive examinations of how people learn from text (e.g. Bransford, Brown, & Cocking, 1999); the interplay among motivation, interest, domain/topic knowledge, and strategic learning over the lifespan (Alexander, 2003; Alexander & Jetton, 2000); analyses of the infrastructure comprehension skills/strategies (e.g. Bruce, Osborn, & Commeyras, 1994; Davis, 1944; 1968); approaches to measuring text complexity (Hiebert, 2011; McNamara, Graesser, & Louwerse, in press; Stenner, Koons, Swartz, in press); and the 2010 ELA Common Core State Standards. The research and perspectives elaborated in each of these sources were then filtered through the steps outlined by Kirsch to develop the model summarized below.

Results/Proposed Model
We propose constructing Text-Task Scenarios (TTSs) designed to tap a variety of purposes for reading, range of texts, and types comprehension tasks, and to take into consideration students’ interests, prior knowledge, and reading strategies as they read specific texts. Before reading each passage or set of passages, a purpose is set that reflects why people at various developmental levels might authentically read. Comprehension items or performance tasks align with the purpose, and indicators of the reader’s prior knowledge and interest are taken into account as performance is quantified and qualified across texts and tasks. Readers’ abilities to engage strategically with a variety of TTSs about which their knowledge, expertise, and interest vary are the basis of their scores. As a result of planned, systematic comparisons, the difficulty of each TTS and factors contributing to difficulty are empirically established. Although some assessments have addressed similar variables, they have rarely been strategically manipulated or has performance been conditionalized. In our model variables are not considered in isolation but as interacting pieces of the comprehension process—and, therefore, part of complexity.

Educational Significance
Text complexity cannot be determined without consideration of the reader, the text, and the task. Constructing new models of comprehension assessment that systematically vary and assess readers while they engage in various purposes for reading will enhance our understanding of text complexity and comprehension difficulty.

Authors

• Sheila Valencia, University of Washington

• Karen K. Wixson, University of North Carolina - Greensboro