How Do Text and Task Interact to Form Complexity?

• In Event: 73.020 - Examinations of the Construct of Challenge: Readers, Texts, and Tasks

Abstract

Purpose: In this study, we explore the interaction of text complexity and task demands across a range of student responses to scenario-based assessments (SBA) administered in a large-scale, grade 4-12 sample.

Perspective: Across grades, students are asked to read greater quantities of increasingly complex texts across multiple disciplinary domains and to perform an ever-widening variety of thinking and learning tasks. Analytically, we can parse elements driving this trajectory of increasing complexity. For one, texts themselves vary in linguistic complexity (e.g., cohesion, syntactic complexity, vocabulary), impacting readers’ ability to form coherent models of the text (McNamara, Graesser, & Louwerse, 2012). Second, the maturity of the content and themes will increase, conditioned on societal norms and expectations. Third, the sophistication of the tasks can vary (e.g., Mosenthal, 1996; Ozuru, Rowe, O'Reilly, & McNamara, 2008).

This complexity can be mediated. Techniques are available to support readers as they face complex texts and tasks (Sabatini, O’Reilly, & Deane, 2013). Students can be given textual aids (e.g., audio, video, definitions) to aid in performing a task targeting a specific proficiency. A complex task can be sequenced into smaller, more manageable subtasks (e.g., asking students to find main ideas and details before writing a summary). Thus, it would seem essential to consider text and task simultaneously when designing learning environments and assessments to inform that learning.

Methods:
Sample: A diverse, multi-school sample of about 1000 students per 4-12th grade completed a pair of SBAs in a cross-grade design intended to enable vertical scale development using an (M)IRT framework. Data has been collected and preliminary item/form analyses have been completed. We anticipate the vertical scaling to be complete in September of 2014.
Instruments: About 20 SBAs of different levels of difficulty were administered in the study. In these SBAs, students are presented with a realistic purpose for reading that requires them to integrate, synthesize, and evaluate a collection of diverse reading materials (e.g., narrative, expository, blog, website). Tasks and activities are sequenced to model complex thinking while simultaneously collecting evidence of partial understanding for developing students. Scaffolding techniques coupled with simulated peer interactions are designed to promote the social nature of reading and the structured nature of learning. In each form, texts range from below to above grade level, based on text complexity formulas.

Analytic Approach: Within and across SBA forms, we examine the interaction of text, task, and students’ developmental level. For example, almost every SBA form begins with a central topical passage about which students write a summary, complete a graphic organizer, and answer comprehension questions. Given these common task types, we can examine how text features drive complexity across ability and developmental levels. Conversely, we can examine how different structuring of tasks may help students mediate understanding of a text.

Significance: The complexity of text and tasks increases exponentially from prekindergarten through 12th grade. Understanding how texts and tasks interact is required to make sensible instructional choices about the texts and tasks that support students’ growth towards college and career proficiency.

Authors

• John P. Sabatini, ETS

• Tenaha P. O'Reilly, ETS

• Laura Halderman, Educational Testing Service