Evaluating the factor structure and measurement invariance of IDELA’s Social-emotional Development Domain across five countries

• In Event: Contextualizing Social and Emotional Learning: Assessment

Proposal

As countries strive to achieve Target 4.2.1 of Sustainable Development Goal (SDG) 4 to track the “proportion of children under 5 years of age who are developmentally on track in health, learning and psychosocial well-being, by sex,” defining and operationalizing constructs to be tracked within and across countries will need to be clarified. Some international bodies have started to outline global metrics to measure learning and development (Meyers, 2004; UNESCO, UNICEF, Brookings Institution, & the World Bank, 2017; UNICEF, 2016). Across all of these, social-emotional development is recognized as an important psychosocial outcome, and one that in early childhood can support children’s transition to school. Yet whether it is possible to measure this construct across diverse countries and cultures, and whether the meaning of the construct is similar across contexts, is a question that remains to be answered. The most commonly used global indicator for comparing children’s well-being across countries is infant/child mortality. While more objective measures of child development, such as infant mortality, are more easily comparable across countries, measures of children’s early developmental skills, including social-emotional skills, may be less comparable across contexts. Yet this question is subject to empirical analysis.

This presentation will assess the measurement properties of the social-emotional development domain of the International Development and Early Learning Assessment (IDELA), a tool developed by Save the Children to measure early childhood development and school readiness. The findings will represent results across five countries including Afghanistan, Bolivia, Ethiopia, Uganda, and Vietnam. First, we evaluate the conceptual model of the social-emotional domain items of the IDELA in each of the five countries separately, and then we address the comparability of the IDELA social-emotional domain across countries. Preliminary findings show that in each country, the social-emotional domain items of the IDELA measure a distinct unique domain of development representing social-emotional skills. This domain is also correlated with the other three domains measured in the IDELA (early literacy, early numeracy, and motor skills) in the expected directions. The psychometric properties of items within each country were quite good overall.

Next, we consider the extent to which the items provide information that is comparable across the five countries. In order to ensure unbiased comparisons of groups (e.g., mean differences), an assessment should satisfy a number of statistical conditions collectively referred to as measurement invariance (e.g., Millsap, 2011). Preliminary results suggest that the social-emotional domain was not invariant over the countries considered in this report. This conclusion did not change when we matched the samples from each country on children’s gender, age, or exposure to formal early child care. We conclude that the IDELA social-emotional domain, taken as a whole, should not be used to make comparisons over countries regarding the state of children’s social-emotional development. We also examine whether measurement bias was due to specific subsets of items or isolated to specific countries. Eight of the 13 items were in fact invariant across at least three of the five countries; two across at least four countries; and three across all five. However, we did not discover any obvious patterns in the items that were invariant. For example, it was not the case that the same country had the fewest invariant items on each domain (i.e., the measurement bias was not plausibly restricted to any one of the countries). Therefore, we conclude that, while there is potential that a core subset of items of the IDELA could be identified for use in cross-country comparisons, this will require careful further research, ideally involving input from experts in international early child developmental as well as additional psychometric analysis.

The results provide initial evidence that the social-emotional items of the IDELA do indeed measure a unique domain of early development skills across multiple countries and contexts. More research is needed to identify a core subset of domains and assessment items that can be used to support cross-country comparisons.


 
References
Millsap, R. E. (2011). Statistical Approaches to Measurement Invariance. New York:
Routeledge.

Myers, R. G. (2004). In search of quality in programmes of early childhood care and education. Background paper for Education for All, Global Monitoring Report 2005. UNESCO.

UNESCO, UNICEF, Brookings Institution, & the World Bank (2017). Overview MELQO: Measuring Early Learning Quality Outcomes. Paris, France: UNESCO.

UNICEF, 2016. https://www.unicef.org/evaldatabase/index_ELDS.html

Author

• Sharon Wolf, New York University