Using TextEvaluator to Analyze the Comprehension Challenges Presented Within Grade 1 Reading Texts

• In Event: 70.021 - Multiple Perspectives on the Complexity of First-Grade Texts Over the Past 50 Years

Abstract

PURPOSE: This study examines variation in the components of text complexity detected in first-grade texts published between 1960 and 2013. Analyses are implemented using TextEvaluator, a comprehensive text analysis system designed to help teachers, textbook publishers, and test developers identify and quantify the sources of comprehension difficulty present in texts targeted at beginning, intermediate and advanced readers (Sheehan, Flor, & Napolitano, 2013).
METHOD: Seven text features designed to measure the ease or difficulty of required cognitive processes are examined. These include: (1) three measures of word difficulty (average word familiarity estimated via the TASA word frequency index, average word familiarity estimated via the ETS word frequency index, and average word length in syllables); (2) two measures of syntactic complexity (average sentence length, and average number of words between punctuation marks); and (3) two measures of the ease of inferring connections across sentences (the referential cohesion measure introduced in Sheehan, 2013, and the Lexical Tightness Index introduced in Flor, Beigman-Klebanov & Sheehan, 2013).
RESULTS: Preliminary results indicate increasing trends in word difficulty and sentence complexity, and slight decreases in referential cohesion and lexical tightness. In all three areas, then, preliminary analyses suggest that the reading challenges encountered by first grade students have increased in difficulty over the past fifty years.
SCHOLARLY SIGNIFICANCE: Implications with respect to the goal of ensuring that all students acquire needed reading skills will be discussed.

Authors

• Kathleen M. Sheehan, ETS

• Michael Flor, Educational Testing Service

• Diane Napolitano, Educational Testing Service

• Chaitanya Ramineni, ETS