A Diachronic Analysis of First-Grade Reading Texts: Shifts and Patterns Detected Using Automatic Analyses

• In Event: 70.021 - Multiple Perspectives on the Complexity of First-Grade Texts Over the Past 50 Years

Abstract

PERSPECTIVE ON TEXT COMPLEXITY. Computational text analysis tools provide a means to examine the fine-grained linguistic properties that are characteristic of particular forms of texts. ALAE (Automated Language Analysis Engine) provides linguistic, semantic, and rhetorical indices from Coh-Metrix (McNamara, Graesser, McCarthy, & Cai, 2014), WAT (the Writing Assessment Tool; Crossley, Roscoe, & McNamara, 2013), TAACO (Tool for the Automatic Analysis of Cohesion, Authors, under review), and TAALES (Tool for the Automatic Analysis of Lexical Sophistication; Authors, in press). Our approach is to examine language from multiple perspectives and using tools that provide information at multiple levels including linguistic, semantic, and rhetorical indices at the word, sentence, and global levels. The purpose of the study is to investigate diachronic linguistic change in a corpus of first grade educational texts taken from seven different time periods (N=492) using ALAE. Our goal is to determine whether and to what degree characteristics of educational texts have changed over time.

METHOD. We calculated 1100 indices of the target texts using ALAE. The indices were checked for normality and multicollinearity.

DATA ANALYSES AND CONCLUSIONS. A MANOVA indicated that 64 indices significantly differed across the seven time points. To investigate the degree to which the texts could be identified as a function of year, we conducted a Discriminant Function Analysis (DFA), which treated year as a categorical dependent variable. An initial analysis indicated that the years 2007 and 2013 were predicted as a single category (with similar characteristics), and these years were thus combined in the subsequent analysis. That DFA indicated that the year of the texts in the total set could be predicted with an accuracy of 66.5%, χ2 (df=25, n=492) = 1050.841, p < .001 (chance for the analysis was 16.67%; baseline was 24.2%; Kappa = .651; see Table 1 for the confusion matrix and results from the LOOCV validation analysis).

These results confirmed that the texts were significantly different in terms of linguistic characteristics as a function of year, indicating that the linguistic properties of standardized educational texts have shifted across the past 50 years. For instance, over time, indices of word sophistication have increased, leading to more difficult and infrequent word choices (see Figure 1). Despite using more sophisticated words, however, texts have grown more repetitive in their word usage (see Figure 2; note M is reverse scaled). Regarding sentence structure, the syntax has grown simpler across time (see Figure 3). Finally, text cohesion has shifted such that connectives are more frequent (see Figure 4); however, there is less overlap in the content between sentences and paragraphs (see Figure 5). Overall, the results of this analysis reveal that properties of educational texts have shifted over time at the word, sentence, and cohesion levels.

Authors

• Scott A. Crossley, Georgia State University

• Laura K. Allen, Arizona State University

• Danielle S. McNamara, Arizona State University