Validation Study of the Remote Assessment of Learning in Palestine

• In Event: Can children’s foundational skills be assessed remotely? Evidence from a multi-country validation study of the Remote Assessment of Learning (ReAL)

Proposal

Palestine provides an important example of an Arab country affected by crisis and possessing many diverse qualities related to place of residence (urban, village and refugee camp), geography (West Bank, East Jerusalem and Gaza strip) and the structure of the educational system (Governmental, private and UNRWA).



A tool like the Remote Assessment of Learning provides an opportunity to sustainably strengthen the quality of the education system. About 1,200 child-caregiver dyads were selected to reflect the demographic diversity mentioned above in addition to the balanced representation of gender and age groups (children 5-14 years). From that sample, sub-samples were assigned to explore inter-rater and retest reliability as well as criterion validity. To enable quality checks, provision of sufficient analysis and comparisons within the cases, a coding system was developed for each record consisting of an abbreviation representing the governorate, place of residence, gender and age group.



Given that 85% of the population in Palestine has access to internet, the high access modality was adopted, and tablets supported field data collection. A theoretical and practical training on the tool was provided and followed by a pilot that enabled the research team to select suitable enumerators. A field data collection plan and quality check plan were developed to support the division of field work into geographic locations and assign each geographic location to one supervisor overseeing a team of field data collectors. A central supervisor then provided overall process management and quality checks. Quality checks for data were done at three stages: field, district and central level. Multiple limitations were identified in the implementation of the tool, including frequent security incidents in the West Bank and Gaza that caused significant delays in completing field work. Additionally, divided authorities in the West Bank and Gaza imposed a longer coordination process in Gaza and limited the teams’ ability to conduct the criterion measure testing in Gaza. These experiences yielded multiple lessons learned. First, international partnerships were important in establishing, validating, and iterating data collection and quality assurance systems. These partnerships are largely feasible and implementable through virtual technologies. Second, the value of this pilot will exceed its testing in the beta phase towards informing gaps in the education system and building valid recommendations for quality improvement effort at the national level. With the contextualization of ReAL in Palestine, the tool is well-positioned for scaling up its use within countries similar to Palestine in terms of culture, language and humanitarian contexts.

Authors

• Ali Nashat Shaar, Palestinian Child Institute

• Nermin Kamal Habaybeh, An Najah National University