50 Years of Text Complexity: A Coh-Metrix Assessment of Changes in First-Grade Texts Over Time

• In Event: 70.021 - Multiple Perspectives on the Complexity of First-Grade Texts Over the Past 50 Years

Abstract

PERSPECTIVE ON TEXT COMPLEXITY. Automated text analysis tools have typically evaluated text complexity using unidimensional readability metrics. Graesser and McNamara (2011) articulated a multilevel theoretical framework that concentrates on deeper comprehension, starting with the word and then up through global discourse cohesion. The framework is compatible with several other models in reading, discourse processing, and education that specify multiple levels of representation and processing components (Kintsch, 1998; Perfetti, 1999).
An automated tool called Coh-Metrix analyzes texts on over 100 measures that tap most levels of the multilevel framework (Graesser et al., in press; McNamara et al., 2014). More specifically, a principal components analysis on 37,520 texts uncovered five major dimensions that account for most of the variability in text complexity (Graesser, McNamara, & Kulikowich, 2011). These include text genre (e.g., narrative versus informational), syntactic simplicity (versus complexity), word concreteness (versus abstractness), referential cohesion, and deep (causal) cohesion. Graesser, McNamara, Cai, Conley, Li, and Pennebaker (in press) recently defined a composite formality score that increases with low narrativity, syntactic complexity, word abstractness, and high referential and deep cohesion: [(referential cohesion+deep causal cohesion – narrativity – syntactic simplicity – word concreteness)/5]. This formality metric has a high correlation with unidimensional metrics of text difficulty as well as other psychological measures that reflect processing difficulty.

METHOD. Coh-Metrix was used to analyze 470 first-grade texts assigned during 1960-2013 in order to investigate changes in text complexity over time. Each text was measured on formality, narrativity, deep cohesion, referential cohesion, syntactic simplicity, and word concreteness. These measures were correlated with year; we also contrasted mean scores before 2000 versus after 2000.

DATA ANALYSES. Correlations between year and the six measures were -.15**, -.40**, .01, -.27**, -.26**, and .39** for formality, narrativity, deep cohesion, referential cohesion, syntactic simplicity, and word concreteness, respectively. The split of texts before versus after 2000 reflected this trend, with one exception. The respective mean scores were formality (-.49, -.79), narrativity (1.33, .76) deep cohesion (-.40, -.50), referential cohesion (1.9, .73), syntactic simplicity (2.36, 2.39), and word concreteness (.40, 1.01). A t-test revealed a significant difference in mean formality t(468) = 7.84, p < .001, narrativity t(468) = 6.84, p < .001, referential cohesion t(468) = 7.95, p < .001, and word concreteness t(468) = 4.49, p < .001. However, there was not a significant difference for deep cohesion t(468) = .84, p = .400 or syntax simplicity t(468) = .29, p =.770.

CONCLUSIONS. The unidimensional measure of text formality suggests texts have become less overall complex or formal with time. However, the multidimentional Coh-Metrix measures paint a more nuanced picture. These results show that first-grade texts have become simpler in some ways, such as containing increasingly concrete language, and more complex in others, including the use of expository style discourse and decrease in referential cohesion.

Authors

• Nia M Dowell, The University of Memphis

• Arthur C. Graesser, The University of Memphis