Screening Secondary Students for Reading Difficulties with the ROAR Comprehension Suite

• In Event: Leveraging Research-Practice Partnerships to Develop Open-Access and Scalable Reading Assessments: Rapid Online Assessment of Reading

Abstract

Objective
This talk will introduce the partnership needs that drove the development of the ROAR-Comprehension Suite, provide an overview of the measures that are currently being developed, and present results from initial validation studies.

Motivation
A nonprofit organization that provides coaching support to secondary schools expressed their need for screening assessments to help guide the selection of interventions for students who were scoring poorly on state-mandated testing. The large number of students at their partner schools made individual assessment impractical. This organization posed a question to the ROAR team: Could an online screener be developed to identify additional instructional targets beyond decoding?

Theoretical Background
While decoding serves as a critical foundation for reading, it is not sufficient for reaching proficiency in reading comprehension (Gough & Tunmer, 1986; Hoover & Gough, 1990; Scarborough, 2001; Kim, 2020). Foundational language skills (such as vocabulary, morphology, grammar and syntax, and inference) also play a significant role in supporting reading comprehension. The ROAR Comprehension Suite enables researchers and practitioners to investigate the development of specific language and cognitive skills that affect reading comprehension. It draws upon the Reading Systems Framework (RSF; Perfetti & Stafura, 2014) and the Direct and Indirect Effects Model of Reading (DIER; Kim, 2020). The RSF model highlights the role of word knowledge in linking word identification and comprehension processes and the DIER model maps how specific language and cognitive skills affect reading comprehension directly, by contributing to meaning construction, and indirectly, by supporting word recognition processes. In accordance with these models, each assessment in the ROAR Comprehension Suite enables measurement of a specific language-related skill: how word parts affect meaning (Morphology), what academic words mean in context (Vocabulary), how words are combined to form sentences (Syntax), and the ability to connect ideas within text and deduce meaning (Inference).

ROAR Comprehension Suite
The ROAR Comprehension Suite consists of four assessments:
1) ROAR-Morphology: an assessment of the ability to apply morphological knowledge within the context of a single sentence.
2) ROAR-Syntax: an assessment of the ability to interpret spoken English syntax and grammar, a digital implementation of the Test for Reception of Grammar (Bishop, 1992).
3) ROAR-Written Vocabulary: an assessment of the ability to understand core academic vocabulary within the context of a single sentence.
4) ROAR-Inference: an assessment of the ability to draw conclusions from text by connecting ideas and deducing meaning within short passages.

Calibration and Validation
A partner district used ROAR-Comprehension to assess 575 students in grades 6-12 who had scored below the 50th percentile in state achievement tests. This talk will present longitudinal findings from this sample as well as analysis of data from an additional 2,500 secondary students who participated in calibration and validation studies for ROAR-Comprehension.

Significance
Most screening assessments primarily focus on decoding skills, leading to interventions that often leave comprehension problems under-addressed (Huebeck, 2023).The ROAR-Comprehension suite fills this gap in response to schools’ needs by providing an efficient, scalable, and culturally responsive tool for assessing component language skills that contribute to reading comprehension.

Authors

• Tonya Murray, Stanford University

• Youngsun Moon, Stanford University

• Jason D Yeatman, Stanford University

• Rebecca Silverman, Stanford University

• Yukie Tomaya, University of California - Berkeley

• Alexander Mario Blum, Stanford University

• Robin Irey, University of California - San Francisco

• Kelly Rose Wentzlof, Stanford University