Literacy as a catalyst for change: Developing an assessment battery to identify key reading precursors in EdTech in developing contexts

• In Event: Linguistic diversity & program strategies: Precursors & potential for foundational learning

Proposal

In recent years, education technology (EdTech) programs have become increasingly prevalent in Sub-Saharan Africa and have been shown to be one of the most effective education interventions in developing contexts (Conn 2017; McEwan 2015; Damon et al., 2016; Rodriguez-Segura 2020). Rigorous research conducted in Africa over the past decade has shown the positive effects of onebillion’s onecourse, an award-winning tablet-based curriculum, on children’s literacy development in both in-school and out-of-school settings (King et al., 2019; Levesque et al., 2020; Levesque et al., 2022; Pitchford et al., 2017). A 15-month randomized controlled trial (RCT) conducted for the Global Learning XPrize with out-of-school children in Tanzania produced effect sizes of 0.46-0.59 in early literacy skills (King et al., 2019). Further, two recent longer-term RCTs with Malawian Standard 2 learners in government primary schools in 2018-19 (8 months) and in 2019-21 (13 disrupted months) produced effect sizes of 0.34 and 0.37 in overall literacy, respectively (Levesque et al., 2020; Levesque et al., 2022). Gains in literacy are also associated with a wide range of positive outcomes for children, including long-term school achievement, school retention, employment, and life expectancy (Duncan et al., 2007; French, 2012; Gilbert et al., 2018). Thus, Edtech programs that support literacy development also have the potential to shape the trajectory of children’s lives.

While onecourse interventions have produced positive literacy effects, further analysis revealed that a substantial portion of children who used the tablet-based curriculum remained non-readers at the end of the intervention periods: 77% after 8 months, 42% after 13 months, and 41% after 15 months. Being a “non-reader” means not being able to read a single word of connected text (words presented in sentences and paragraphs), as measured by the Early Grade Reading Assessment (EGRA; Dubeck & Gove 2015). A recent exploratory study conducted by Bardack and colleagues (2023) examining divergent patterns in children’s reading progressions at a peri-urban Malawi primary school showed that children’s working memory skills emerged as the strongest, unique predictor of high versus low progress in reading while controlling for other factors. These findings align with prior research conducted on a sample of first-grade learners in the Netherlands who participated in a tablet-based word reading efficiency game intervention found that multiple reading precursors that were related to children’s in-game accuracy (working memory, phonological awareness and letter efficiency) and in-game efficiency (phonological awareness, and rapid automatic naming [RAN]) predicted higher growth in reading efficiency (van Utter et al., 2022). Yet, more research is needed to better understand how these critical reading precursors may co-occur to support children’s reading progress in tablet-based curriculum interventions in a developing context.

The current study presents data from a pilot study designed to assess the reliability and validity of a battery of assessments that will be used to replicate and extend prior research identifying important predictors of reading progress (Bardack et al., 2023; Daneman & Carpenter 1980; Dubeck et al., 2017; Engel de Abreu et al., 2014; Follmer 2018; van Utter, 2023). The battery includes three assessments that test known predictors of early literacy and that have been previously administered in Chichewa in Malawi: a digitized spatial span measure of working memory (Pitchford & Outhwaite, 2016) as well as an initial syllable measure of phonological awareness and a rapid automatic naming (RAN) measure of processing speed (Dubeck et al., 2017). We conducted the pilot test on a subsample of learners from a large, urban primary school that is participating in implementation research to prepare for a national scale-up of onebillion’s tablet-based curriculum program in Malawi beginning in Fall 2023.

To select our subsample, we identified learners who had complete data for the onecourse embedded e-assessment (i.e., the onetest), which was administered at baseline in February 2023 and at midline in May 2023. Among these learners, we purposively selected 60 children who showed a range of literacy achievement patterns from baseline to midline to participate in the pilot study. During the endline administration of the onetest in early July 2023, all pilot participants completed the battery of three assessments: (1) digitized forward span task; (2) an initial syllables task; and (3) a RAN task. In addition to evaluating the reliability of these three assessments, we will run bivariate correlations to test construct validity by comparing children’s performance on each assessment with their performance on the literacy and numeracy assessments from the onetest at baseline, midline and endline.

With the pilot concluding in July 2023, we will present our full findings at the 2024 CIES conference. Our results will be used to: (1) inform future research on factors influencing non-progress in reading in developing contexts; (2) determine the feasibility and utility of a battery of assessments to identify children who are at risk for non-progress in reading and; (3) contribute to continuous improvement of the onecourse software during the upcoming nationwide scale-up of the tablet-based curriculum through the Building Education Foundations through Innovations & Technology (BEFIT) program. As the BEFIT program scales to provide access to child-directed, tablet-based learning for all primary school children in Malawi, this research will contribute to ensuring that children who are at-risk for non-progress in reading can benefit from the opportunity to develop foundational literacy skills that can be a catalyst for change in their lives.

Authors

• Sarah Bardack, Imagine Worldwide

• Margaret (Peggy) M Dubeck, RTI International

• Karen Levesque, Imagine Worldwide

• Celeste Lopez, Imagine Worlwide