Performance of the International Development and Early Learning Assessment (IDELA) when used with children with functional difficulties in Kenya

• In Event: Promising approaches to generating inclusive data for children with disabilities: the role of learning assessment tools

Proposal

Background: There is an urgent need to scale up early childhood development and education (ECDE) services in low- and middle-income countries (LMICs), particularly for children with disabilities. To do this effectively, tools that reliably measure early childhood learning and development for all children, including those with disabilities, are essential. The IDELA is a standardised assessment of early childhood learning and development, validated for use in LMICs, and including guidelines for use with children with a range of functional difficulties. While provision of guidelines for inclusive assessment is crucial, little is known about the impact of inclusive assessment on the tool’s performance or implications for appropriate data analysis. We use data from IDELA assessments of young children with functional difficulties in Kenya to explore the tool’s performance when used with this group. We draw on learnings from a qualitative study of experiences in administering inclusive assessments, and our experiences working with the data collected, to reflect on analysis and interpretation of this data.
Methods: We use baseline data from an impact evaluation for an inclusive ECDE intervention in two regions of Kenya: Homa Bay County, and Kakuma Refugee Camp and surrounds. Baseline data includes 1615 IDELA assessments of young children, aged 2 to 9, recruited from 18 pre-schools during Sept/Oct 2021 and May/June 2022. Of these, 248 were completed with children with functional difficulties, identified using the UNICEF/Washington Group Child Functioning Module with standard cut-offs.
We examined the proportion of missing/skipped responses, overall and within each domain, for children with and without functional difficulties. We explored associations using linear regression models adjusting for age and accounting for clustering within schools. To understand whether items behaved consistently for children with and without functional difficulties, we assessed internal consistency of each IDELA domain using standardized Cronbach’s alpha.
Results: We found no evidence of significant differences in proportion of missing/skipped responses on the basis of functional difficulty status. All IDELA domains showed good levels of internal consistency (standardised Cronbach alpha 0.8-0.9) for both children with and without functional difficulties. Assessors reported some challenges in adequately documenting use of adaptations during data collection. Individual children’s difficulties were very diverse, requiring tailored adaptations. Recording these adaptations was time consuming and challenging to do while busy with a child, but recall was a challenge in doing this after the assessment. During analysis, qualitative notes did not always provide adequate clarity.
Discussion: These results are consistent with qualitative feedback that IDELA could be completed with most children with functional difficulties without omitting items or making substantive adaptations. There is no immediate evidence for general adjustments to analysis processes in most cases, although further research into the impact of specific adaptations on scores would be useful. Additionally, use of IDELA with disability-inclusive adaptations will be strengthened by development of guidelines for recording use of disability-specific adaptations, and further research into when, and how, data analysis should be adjusted.

Authors

• Sheru W. Muuo, Sightsavers

• Anita Jeyam, Sightsavers

• Julia de Kadt, Sightsavers

• Veronica M Stapleton, Sightsavers

• Elena Schmidt, Sightsavers