Simplifying Middle-Grade Texts Using AI Extensions: Impacts on Text Vocabulary and Cohesion

• In Event: AI in Literacy Education: Imagining the Future While Building on Proven Practices

Abstract

Objectives: This study examined how AI-generated simplifications affect syntactic complexity beyond sentence length and how they impact vocabulary richness. AI browser extensions allow teachers to adjust reading levels quickly. For example, a text designated to be at an eighth-grade reading level can be simplified to a 6th- or 4th-grade level. Such tools use readability algorithms that manipulate word frequency and sentence length but their effects on deeper text features, particularly cohesion, remain unclear.

Guiding Questions:
1. How does the cohesion of an 8th-grade passage change when simplified, as measured by: a) referential cohesion, b) modifiers per noun phrase and c) left-embeddedness (the grammatical structures that precede the main clause or verb)?
2. How does vocabulary change across simplified passages, as measured by: a) concreteness, b) age-of-acquisition, and c) causal verbs?

Theoretical Framework: Readability formulas (Klare, 1975) predict difficulty using sentence length and word frequency but ignore cohesion. Cohesion, however, is an essential component for comprehension, especially for students with lower background knowledge and skill (McNamara et al., 1996). Text cohesion is the degree to which the linguistic elements repeat or refer back to previous ideas using pronouns, repeated words, and/or connective words. Text cohesion helps readers maintain a thread of comprehension and reduces the inference burden on the reader.

Methods and Data Sources: The study analyzed 150 eighth-grade texts in the range of 900–1150 Lexiles and averaging 1199 words in length. This set of texts was simplified to 6th and 4th grade levels using Brisk Teaching, an AI-powered browser extension that includes a text modification tool using automated readability algorithms that manipulate sentence length and word frequency.
The 450 texts (150 at each of the 4th, 6th, and 8th grade levels) were evaluated with three vocabulary variables (age-of-acquisition, concreteness, causal verbs) and three cohesion variables (left-embeddedness, referential cohesion, modifiers/noun phrase). Repeated measures analyses compared levels.

Results: As texts were simplified from grade 8 to grades 6 to 4, three trends occurred in vocabulary: a) increases in concreteness; b) increases in causal verbs; and c) decreases in age of acquisition estimates. Essentially, vocabulary metrics all reflected words that became easier when simplified using Brisk.
On some measures of cohesion, simplifications likely made texts easier to comprehend: both left-embeddedness and number of modifiers per noun phrase decreased. Simplification also resulted in decreases in referential cohesion, but this decrease in overlap between stem and argument is likely to impede, rather than facilitate comprehension.

Scientific Importance: While vocabulary simplification aligned with expectations—words became more concrete and accessible—AI-modified texts also became shorter, potentially limiting reading stamina. More critically, cohesion declined in ways that may hinder comprehension. Unexpectedly, referential cohesion dropped, reducing repetition and overlap that support comprehension—particularly for less-skilled readers. These findings suggest that, while AI tools simplify surface-level features, they may inadvertently compromise deeper elements essential for comprehension and learning.

Authors

• Heidi Anne E. Mesmer, Virginia Tech

• Elfrieda H. Hiebert, TextProject