Appraising the Qualities of Two Online Reading Programs: Affordances, Architecture, and Functionality

• In Event: 66.017 - Content Features in Digitized Stories That Promote Young Children's Vocabulary and Story Comprehension

Abstract

Purpose
As the supply of online reading materials increases, pressure mounts to use these resources in students’ reading instruction and to improve fiscal efficiency. Little is known, however, about the quality effectiveness of these materials even as ESSA initiates funding their adoption. This study appraises the product qualities of materials contained in two commercial online reading programs.

Perspectives
The look and feel of curriculum materials in the teaching of reading faces complex, if not troubling waters, caught up in the digital turn from print to pixel, page to screen, hardbound to electronic book on mobile devices. At scale, we cannot expect teachers to both evaluate and use online reading materials on a daily basis. Early steps have been taken to vet e-books and apps by expert groups (Children’s Technology Review), although methodologies lack scientific rigor (Hirsh-Pasek, Zosh, Golinkoff, Gray, Robb & Kaufman, 2015). Few technical tools are available for the critical analysis of e-books/apps at the instructional design level (Roskos, Brueck & Widman, 2009). Consequently, the extent to which e-texts represent principles of e-learning (Clark & Mayer, 2008) and learning to read (Snow, Burns & Griffin, 1998) is ill-defined, making it difficult to judge whether digital materials are “good enough” for reading teaching.

Method
A mixed methods approach was used to examine program materials. Raz/ Raz-Kids and Storia online materials constituted the sample. A random sample of e-book titles, representing 10% of each program, was drawn (n=196). Affordances of e-book collections and dashboards were assessed; a subsample of screen pages was analyzed vis-a-vis multimedia learning principles; the total program package was evaluated for functionality.

Results
A significant difference was found between e-book collections in Ease of Use (p <.0..05) and Multimedia (p < .05) elements in favor of Raz/Raz-Kids. There were no significant differences in children’s literature elements. Neither dashboard fully met quality criteria. Principles of dual coding, synchronicity and voice quality were met whereas principles of contiguity, assistance and naviation were weak. The difference between programs on the assistance principle was significant (p<0.05). As a package, Raz outperforms Storia offering a fuller range of material types at sufficient levels for instruction. In both programs, teaching guidance met >50% of criteria in comprehension and vocabulary domains; Raz met >50% of criteria in the oral language domain whereas Storia met <40% of these criteria. In word-level skill domains, Raz met a greater percentage of fluency (42%) and phonics (50%) criteria than Storia at 29% and 32% respectively. Both programs met <40% of criteria in phonemic awareness and writing domains; only Raz met any differentiation criteria (<20%). Broadly both programs are stronger in meaning-level best practice domains than word-level domains.

Significance
The appraisal provides evidence of the qualities of two popular program products as well as an analytic framework for evaluation. At the digital turn, online reading programs provide an efficient option, but more collaborative design work among developers and educators is needed to improve their potential effectiveness.

Author

• Kathleen A. Roskos, John Carroll University