Leveraging Voice AI for Scaling EGRA-type Assessments in low-resource language in the Global South

• In Event: Leveraging Voice AI (Artificial Intelligence) to assess early grade reading skills and support learning

Proposal

Early Grade Reading Assessments (EGRA) were developed as a simple, short and inexpensive assessment of foundational learning and numeracy skills. However, conducting EGRA at scale, to capture a snapshot of a country’s learning levels, involves a significant and expensive logistical effort, often largely financed by donors and some level of Government contribution. The time-intensive, one-on-one administration can also disrupt classroom activities, and it is too time-consuming for teachers to conduct these assessments regularly.

AI (Artificial Intelligence) presents an opportunity to lower the costs of collecting and marking EGRA tests at scale. With features like voice recording and automatic marking, results can be fed back into the system and provided directly to teachers and students, making the assessment process more efficient and impactful.

While a fully AI-driven EGRA system that can reliably measure all subtasks has yet to be developed, several innovations in this space show advances in the fine-tuning of Voice AI models for children’s voices across several languages with substantial potential. These include Afrikaans (South Africa), Bahasa (Indonesia), Chewa (Malawi), Filipino (Philipines), Kiswahili (Tanzania), Kamba (Kenya), IsiXhosa (South Africa), Siswati (South Africa), and Sepedi (South Africa). For instance, the EGRA-AI initiative assesses letter sounds and single-word reading in Sepedi and IsiXhosa, and RTI’s self-administered CoBRA test pilot in the Philippines measures oral reading fluency in English and Filipino. These initiatives show that reading assessment powered by AI can reduce the logistical burden of assessing children. However, they also underscore the need for further development of an AI technology that can reliably assess all subtasks, so that it can be used as a tool to assess learning.

However, challenges remain in fully developing AI technology for reading assessment. A major hurdle is the difficulty in obtaining children’s voice data. For example, the EGRA AI pilot encountered difficulties in planning data collection for obtaining a sufficient balanced sample of correctly and incorrectly spoken items for training the models, without knowing in advance which items would be skipped or prove challenging for children. Meanwhile, RTI outsourced the speech recognition service, finding the licensing costs unsustainable to measure learning at scale.

Another challenge is data privacy. Even when projects successfully gather data, it is often not made publicly available due to high costs and the complexity of ensuring safe use of children's data.

This presentation will provide a simple framework and actionable guidance to help developers build on existing resources, and make informed choices in the collection and use of children’s voice data. This will reduce the costs associated with adding new languages.

Author

• Romana Kropilova, Fab Inc