- Browse/Search the Program
Search
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Committee or SIG
Browse By Session Type
Browse By Keywords
Browse By Geographic Descriptor
- Information Menu
Search Tips
- Navigation and Settings Menu
Personal Schedule
Change Preferences / Time Zone
Sign In
Remotely Assessing Foundational Skills of 5-14-Year-Old Children: A Six-Country Psychometric Evaluation of the Remote Assessment of Learning (ReAL)
Wed, March 26, 1:15 to 2:30pm, Palmer House, Floor: 3rd Floor, Salon 6Proposal
Sustainable Development Goal (SDG) 4 commits the global education community to ensuring all children achieve essential learning and development outcomes. This requires investing in the most educationally marginalized children: the estimated 250 million children out of school and the millions more not learning adequately, especially those in conflict and crisis areas (UNESCO, 2023). By 2030, the UN estimates that 300 million students will lack essential basic numeracy and literacy skills (United Nations, 2023). Global stakeholders emphasize the need for contextually appropriate, psychometrically sound measurement tools to support quality education in crisis contexts (Tubbs Dolan, 2019). These tools provide critical data for evidence-based decision-making within education systems.
While face-to-face learning assessments have seen growth, very few remotely administered assessments using basic or advanced technologies exist. Developing and testing such assessments is urgent, as educators in crisis settings struggle to evaluate children’s academic and social-emotional skills when face-to-face assessments are not possible due to lockdowns, school closures, or other crises.
The Remote Assessment of Learning (ReAL) tool, a remote assessment designed for children aged 5–14-years, measures literacy, numeracy, and social-emotional learning (SEL) skills for monitoring and evaluation. The literacy domain contains an oral language module (with expressive vocabulary, listening comprehension and retelling a story sub-domains) and a reading module (with letter/letter sound identification, common word identification, sentence-level comprehension and oral passage reading sub-domains). The numeracy domain contains five sub-domains (one-to-one correspondence, number identification, addition, subtraction, and word problems). The SEL domain contains six sub-domains (self-concept, relationships, perseverance, stress management, empathy, and conflict resolution). The ReAL assessment is adaptable to various technological contexts, offering three administration modalities (High Access, Low Access, or Caregiver-reported) depending on child accessibility, materials availability, and the type of phone available to caregivers (Save the Children, n.d.).
The ReAL Network, including Save the Children, local academic partners, and a global consortium of practitioners and researchers, conducted a cross-cultural validation study targeting hard-to-reach children in seven low- and middle-income countries (LMICs). This study addressed three main issues: (1) the lack of rigorous evidence on remotely administered assessments in LMICs (Angrist et al., 2022); (2) a gap in translating research into practice due to limited resources (Shumba et al., 2021); and (3) that most measures used in LMICs are developed by or drawn from educational science in high-income countries (HICs) and the extent to which they provide valid and reliable data in crises has been called into question (Bartlett et al., 2015; Dowd et al., 2019; Halpin & Torrente, 2014).
The study assessed the psychometric properties of the High Access modality of the ReAL tool, focusing on inter-rater reliability, factor structure, item difficulty, criterion validity, and test-retest reliability. Results show moderate evidence that ReAL is a valid and reliable measure for literacy and numeracy sub-domains, with less robust evidence for social-emotional sub-domains. The validation study results, along with revisions to the measure, will be presented, and the implications for future tech-enabled assessments of learning and development will be discussed.