Tablet-based Self-administered EGRA/EGMA Development and Practical Lessons

• In Event: Reaching remote or independent learners through tech-enabled assessments of learning and development

Proposal

This presentation shares practical experiences and findings from nearly 3 years of ongoing research and development of two assessments, known respectively as the Self-Administered Early Grade Reading Assessment (SA-EGRA) and the Self-Administered Early Grade Mathematics Assessment (SA-EGMA) in English and Chichewa. The assessments are deemed “self-administered,” because children complete the assessments independently in response to instructions and stimuli imbedded in the tablet-based software, RTI’s open-source data collection platform, Tangerine. The assessments are suitable already for young children. Adults typically supervise the organization and conduct of the assessment but do not need to sit one on one with the child as is the case for the traditional EGRA/EGMA.

The rigorous assessment development effort started in late 2021 with an English-language version specific to Ghana. This version was later adapted to English language versions for Liberia and Sierra Leone, and recently for Chichewa in Malawi. Currently, the team is working in Tanzania on a Kiswahili version. To start with, the team reviewed existing EGRA/EGMA instruments for relevant assessment tasks and items. The team then adapted these items to the self-assessment format and specific country context, and, working with local researchers, conducted user testing with up to 40 students in the early grades. User testing helped refine the design of the user interface of the assessment on the tablet, the integration of audio prompts, and the presentation of individual tasks and items.

Revised tools were then field tested with up to 500 students, depending on country. The research team analyzed the data from the field tests using Item Response Theory and Factor Analysis and used the results to revise assessment tasks and items. A pilot test, with 400-500 students depending on country, followed, that assessed internal consistency and reliability of each assessment as well as concurrent validity with the traditional EGRA/EGMA. Lastly, the researchers explored test-retest reliability, to determine if students perform consistently across timepoints. Throughout, the tools showed acceptable internally consistency, reliability, and concurrent validity against standard EGRA/EGMA tools. The presentation will share more detailed results from the psychometric analyses.

Furthermore, the presentation will share practical lessons from applying the tool as part of the tablet-based foundational literacy and numeracy curriculum programs implemented by Imagine Worldwide in sub-Saharan Africa. Illustratively, in these contexts, the tablet-based SA-EGRA and SA-EGMA not only required shorter and less in-depth training of enumerators but also showed to produce valid and reliable results at about one quarter of the cost of standard assessments.

The presentation will conclude with an outlook on future plans, including for Artificial Intelligence (AI)-integration that would allow for the assessment of expressive language and fluency. Use of the SA-EGRA and SA-EGMA is free. Presenters will share how to access the Tangerine installation files.

Authors

• Abraham Bahlibi, Imagine Worldwide

• Jennifer Ryan, RTI International