Exploring possibilities for a short-form IDELA tool

• In Event: 2-101 - Developing and Validating Multi-Dimensional Early Childhood Development Measures for use Across Countries and Cultures

Integrative Statement

IDELA is a validated measure of children’s early learning and development skills and has been used primarily for project monitoring and impact evaluations in over 50 low-resource contexts around the world. IDELA was designed to collect rigorous and meaningful data with a relatively straightforward and simple-to-administer survey. On average, administration takes 30-35 minutes and can be undertaken by local community members. While the tool has been useful for a wide variety of purposes, the time and cost of data collection is substantial and can be a barrier to usage in large scale or national data collections. This presentation explores the possibilities for creating a “short form” to provide a level of rigor between population-level caregiver-reported surveys such as the ten-question Early Childhood Development Index (ECDI) and the full form of IDELA.

ECDI, created by UNICEF, is the only population level ECCD measure for children older than three years. The ECDI includes 10 parent-reported items that capture information about literacy, numeracy, social-emotional, and physical development for 3–4 year olds (e.g., Does (name) know the name and recognize the symbol of all numbers from 1 to 10?). Children are considered to be ‘developmentally on track’ if the number of ‘Yes’ responses given by their parents is sufficient in three of the four domains. The ECDI has been administered for more than 160,000 children in over 60 countries, and the data has been used to inform global knowledge about child development in LMICs. However, researchers have noted that the ECDI in its current format is lacks depth of information about children’s learning and development due to its limited number of parent-reported questions and restricted age range to 36–59 months (McCoy et al., 2016). A shorter form of the IDELA tool could serve to provide more detailed and reliable information about young children’s learning and development.

This presentation attempts to answer the question “What IDELA sub-tasks best predict the overall IDELA score across contexts?” We use a large database (N = 20,000+) of IDELA assessments from 39 contexts to explore this question. We first estimate the following model:
IDELA_total=β ̂_0+β ̂_(subtask_1 )… β ̂_(subtask_n )+β_dataset1… β_dataset39

for all possible combinations of up to n = 5 subtasks (a total of 55,454 regression models) including dummy variables to control for intercept differences between datasets.

The overall IDELA score is the average of domain scores, which are in turn the average of subtask scores, so by design all subtasks should be positively correlated with the overall IDELA score. However, the explanatory power of different subtasks and model specifications produces highly variable results. Using the dataset of 55,454 regression models, we examine the reliability of different subtasks to predict overall IDELA by summarizing the fit (R^2) of models that include each subtask. The results of this analysis suggest that, by judiciously selecting four subtasks, we can reliably explain over 80 percent of variation in total IDELA score. We conclude by examining the most consistently predictive subtasks and suggest routes to further explore these subtasks (e.g. Item Response Theory).

Authors

• Jonathan Seiden, Save the Children
  Presenting Author

• Peter Halpin, New York University
  Non-Presenting Author

• Lauren Pisani, UCL
  Non-Presenting Author

• Amy Jo Dowd, Save the Children
  Non-Presenting Author