Validation Study of the Remote Assessment of Learning in Mozambique

• In Event: Can children’s foundational skills be assessed remotely? Evidence from a multi-country validation study of the Remote Assessment of Learning (ReAL)

Proposal

The pilot for the Remote Assessment of Learning (ReAL) in Mozambique took place from April 17 to July 16, 2023. It involved a sample of about 520 children, both in and out of school, across two age/grade groups. Age and grade groups included children aged 8-9 years old targeted for grade 3 and children aged 11-12 years old targeted for grade 7. Of the total sample, 20% of the cases were randomly selected as an inter-rater subsample, and another 20% were chosen for a second assessment approximately four weeks after being administered the ReAL beta pilot (i.e., test-retest subsample). Approximately 4-6 weeks (about 1 and a half months) after the initial assessment, another randomly selected 20% of the children also completed a set of criterion measures (EGRA, EGMA, survey of social-emotional development). The assessment tool was adapted to align with Mozambique's educational curriculum and language, involving collaboration between teachers and researchers at the Faculty of Education, Eduardo Mondlane University, and supported by Save the Children in Mozambique.

The pilot study targeted four rural schools (ECP Mieze, Nacarramo, Nanlia and Impire) in the Cabo Delgado, Metuge District. Through the high access administration modality, conventional phone calls and hard-copy assessment-specific learning materials were used. Verbal informed consent was obtained from the primary caregiver, along with assent from the children.

The primary aim of the pilot study was to evaluate the feasibility and validity of remotely assessing children's foundational skills using ReAL. The research focused on assessing the effectiveness of ReAL in capturing the foundational skills of Mozambican children, particularly in the domains of literacy, numeracy, and social and emotional learning. The study also aimed to identify contextual factors that might influence the results and determine the extent to which ReAL can be a reliable measure of learning outcomes in the Mozambican education system.

In the Mozambican context, ReAL has the potential to inform education policies and interventions with timely and accurate data on children's learning outcomes, enabling evidence-based decision-making. By identifying areas for improvement, ReAL can contribute to the development of targeted interventions and the allocation of resources to support children's learning needs. While remote assessments like ReAL have the potential to revolutionize education by offering scalability and to reach students in various educational settings, some challenges remain: equitable access to technology, competence of data enumerators, data management, security and privacy concerns, technological infrastructure, standardization and adaptation, teacher and parent involvement, ethical considerations and continuous improvement. These all need to be carefully considered when exploring the broader implementation of remote assessments in education.

In conclusion, the pilot process of ReAL in Mozambique has provided valuable insights into children's foundational skills of learning and the potential of remote assessments in the country. The results have the potential to inform decision-making processes in education and emphasize the importance of leveraging technology for data-driven interventions. Moving forward, it is essential to further explore the use of ReAL and similar remote assessment tools in education, considering the unique challenges and opportunities presented by each context.

Authors

• Adriano Simao Uaciquete, Universidade Eduardo MondlaneGhent University

• Belmiro Novele

• Dyson Joe Likomwa, Save The Children International

• Nelson Zavale, Save The Children International