Modifying IDELA for Remote Use: Implications for Assessing Child Development in Challenging Circumstances

• In Event: Teaching and learning experiences during COVID-19

Proposal

The COVID-19 pandemic has challenged the way we conduct research. For some modes of data collection, such as interviews, there is a ready (if not perfect) analog: face-to-face becomes phone-based; paper and pen surveys move online. Others, such as direct assessments of child development, prove more challenging. Despite the challenges, there is a benefit to being able to conduct child assessments entirely remotely. Even in the absence of a pandemic, some areas are difficult to access due to conflict, natural disasters, or other crises. Others are prohibitively expensive to access due to their remoteness. Given reliable tools, monitoring and evaluating programs in such circumstances may best be done remotely. This paper presents a case study of how we adapted and piloted a remote/phone-call based version of the International Development and Early Learning Assessment (IDELA) in a highly challenging environment and offers a psychometric assessment of how the remote IDELA performed and what changes might increase its reliability and validity in this context. Findings are critical for future early childhood development programming and research in Lebanon and in other contexts where in-person assessments are not feasible but remote assessments present their own (often daunting) challenges.

The IDELA is a tool for assessing 3- to 6-year old early numeracy, early literacy, motor, and social-emotional development that has been used in over 75 countries and undergone multiple cross-country validations (e.g. Halpin et al. 2019; Pisani, Borisova, & Dowd, 2018). It has traditionally been conducted one-on-one and in person. The International Rescue Committee (IRC) in Lebanon has used the in-person IDELA for many years to monitor and evaluate it’s ECD programs’ contributions to child development. During the COVID-19 pandemic, in-person programming and data collection was no longer possible. Therefore, in advance of an evaluation of an early learning program conducted remotely via Whatsapp messaging and video chats, the IRC and New York University (NYU) Global TIES collaborated to adapt IDELA for smartphone-based data collection with caregivers and children. Items needing resources families might not have on hand were modified (e.g. substituting an e-book for a hard copy book) or dropped (e.g. dropping the jigsaw puzzle item). The resulting 26-item tool was piloted with 3- to 6-year-olds in the Akkar, Bekaa, and Tripoli regions of Lebanon using WhatsApp video calls. Data is being collected in October and November 2021. Based on prior piloting, assessments are anticipated to take 30-40 minutes each on average.

Our final sample is anticipated to include 535 children: 97.6% Syrian Refugees, 2.2% Lebanese, and 0.2% stateless. 189 were finishing KG1 (first year of kindergarten in Lebanon, for children aged 3-4); 244 were finishing KG2 (second year, children aged 4-5); and 102 were finishing KG3 (third year, children aged 5-6). Children were sampled from those participating in the IRC’s 2021 remote early childhood education programming.

We will first present on what changes were made to the IDELA in order to adapt it both to remote data collection through phone calls and the local context (e.g., images of bikes were changed to be pencils). We will then walk through the remote data collection process focusing on the role of caregivers (who have no role in in-person IDELA assessments) and steps taken to reduce bias and caregiver coaching of their child. Other challenges, such as connectivity and phone camera issues as well as electricity outages, and how we worked around them will be discussed as well. We will close by presenting psychometric findings of the remote IDELA and recommendations for adaptations and protocols for future remote use in Lebanon and globally.

Finding ways to remotely assess child development is critical to improving the quality of and assessing early childhood programs in difficult to access contexts. Even in the absence of a pandemic, some areas - especially those facing multiple crises such as Lebanon is currently - are difficult (or too costly) to access for in-person data collection. These settings often also present additional challenges to remote data collection, such as electricity shortages and limited internet access. And yet, direct assessments still present the best way to monitor and support child development and improve early childhood programming. As such, it is critical that we better understand how to conduct them under a wide array of conditions, including fully remote programming and data collection.

Authors

• Kate Schwartz, New York University

• Lina Torossian, International Rescue Organization

• Duja Michael, NYU Global TIES for Children

• Diala Hajal, International Rescue Committee

• Katie Maeve Murphy, International Rescue Committee