Validation Study of the Remote Assessment of Learning in Cambodia

• In Event: Can children’s foundational skills be assessed remotely? Evidence from a multi-country validation study of the Remote Assessment of Learning (ReAL)

Proposal

The COVID-19 pandemic caused Cambodian school closures in 2020 and 2021 that interrupted student learning for almost 300 days. The COVID-19 pandemic in Cambodia reflected the lack of distance learning readiness, including instructional materials, methodologies, and learning assessment tools and mechanisms. During the school closures, the use of distance learning was adopted and adapted to the context of schools, teachers, and students; 69% of students and 95% of both teachers and school directors used distance learning and teaching methods. However, there was no remote tool or method available to assess learning levels of students despite around three quarters of children and caregivers (72%) having access to mobile phones. Therefore, it is necessary to have a valid and reliable remote tool for the Ministry of Education, Youth and Sport (MoEYS) and education stakeholders to conduct learning assessments during the emergency situations, particularly when face-to-face interaction is not appropriate.

Save the Children Cambodia and relevant departments from MoEYS jointly conducted the beta pilot of ReAL to assess literacy, numeracy, and psychosocial outcomes of students using the high access modality developed by Save the Children International, Save the Children USA, Yale University, and the Free University of Berlin. The assessment aimed to test validity and reliability of learning and psychosocial outcomes remotely using the high access version. A total sample of about 1000 children aged 5 to 12 years, together with their respective caregivers, were randomly selected to participate in this study in three districts (Kampong Trolach, Rolea B'ier and Samaki Meanchey districts) of Kampong Chhnang province, Cambodia. A quota sampling approach was employed to approximate a representative sample in terms of district/location, age group, and child sex. In addition, randomly selected subsamples of 20% of dyads were assessed by two enumerators (inter-rater subsample), 20% of randomly selected children were administered the criterion measures using standard literacy and numeracy tests of MoEYS, and a random 20% of the total sample completed ReAL at a second measurement occasion (approx. 4-6 weeks after the first occasion).

In this paper, we provide evidence for the reliability and validity of the ReAL high-access tool. We assess the inter-rater and test-retest reliability, the structural validity utilizing confirmatory factor analysis, the criterion validity by assessing the correlations between our domains of learning and the standard tool of MoEYS, and the measurement invariance by key socio-demographics. The results will inform relevant government agencies, in particular MoEYS, and development stakeholders to adopt and adapt the remote assessment tool. Benefits of adopting ReAL include cost savings and efficient data collection in the context of low government budget for assessment purposes. In addition, the remote tool will help build the monitoring scheme of the MoEYS’s Education Quality Assurance Department (EQAD) which oversees the learning assessment affairs of the ministry. The tool will also be used in the context of hard-to-reach children, particularly the floating villages and schools in Tonle Sap Lake of Cambodia.

Authors

• Rada Khoy, Save the Children International

• Borin Srey, Save The Children International

• Sakem Kong, Save The Children Cambodia

• Puthichan Pha, Save The Children Cambodia

• Sorachana Seng, Save The Children Cambodia

• Sithon Khun, Save the Children

• Sarin Sar, Ministry of Education, Youth and Sport