Developing Contextually Relevant Socioemotional and Math Assessments: Evidence from Colombia Mass Media Evaluation Pilot Study

• In Event: Evaluating educational mass media for children affected by conflict and displacement: Lessons learned from Jordan and Colombia

Proposal

More than 460,000 of the displaced populations in Colombia, a country with the largest number of Venezuelan refugees, are school-aged children with limited to no access to education including socio-emotional learning and science/mathematics education (Human Rights Watch, 2022; International Rescue Committee, 2020). In this context, Watch, Play Learn video blocks target four to five-year-old children through a randomized experiment in Colombia with a focus on Venezuelan migrants living in Colombia and Colombian nationals in Barranquilla and Soledad with a high concentration of Venezuelan migrants. To inform the study design, we conducted a quantitative measurement pilot study in two phases (phase 1, n=48, phase 2, n =27 households) to ensure reliable measurement of socioemotional and math skills among children and household characteristics. The sample comprised a population of four-year-old children and their caregivers (mostly mothers) in Colombian and Venezuelan households in Santa Marta, Colombia. In addition, qualitative data was collected to identify program neighborhoods and develop a cost study questionnaire. Focused group interviews were conducted with Colombian and Venezuelan community leaders to determine effective implementation strategies utilizing community spaces, and the feasibility of the delivery of short video content over WhatsApp was examined in partnership with a local tech agency.

The findings from the pilot fall under two main categories. First, the survey results fall into three major areas: (1) child direct assessments (CDA) focused on children’s socioemotional skills demonstrate good variability in responses, especially for expressive and receptive emotional skills. However, emotions such as frustration and fear were difficult for children to identify, offering an opportunity for the PtL program to teach children about these emotions; (2) basic numeracy skills assessments demonstrated a good amount of variability in math skills such as counting, shape recognition, and relational concept questions whereas concepts like pattern recognition, and cardinal and identity principles generated high percentages of incorrect responses with low levels of variance. New items from IDELA and the Birthday party assessment were added to emphasize PtL curricular concepts such as size, height, width, spatial relations, and counting for smaller and larger numbers; (3) Caregiver survey focused mainly on demographics, caregiver engagement, and perceptions of children’s developmental status. The indices showed acceptable to moderate levels of reliability and variance. Items with low and zero variance and loadings below 0.4 were dropped in the pilot data analysis. Second, the qualitative findings from the second phase of the pilot study showed that all caregivers accompanied their children while they watched Sesame-created content videos over Whatsapp. Almost all parents reported needing no extra support from the program team for watching the videos and were comfortable meeting with the enumerators in community spaces. Overall, the findings were utilized to modify CDA, CG, and cost questionnaires for culturally, age-appropriate questions, easily comprehensible by both children and caregivers as well as effective program delivery and data collection strategies. The findings have implications for early childhood development programs and policies, especially concerning socio-emotional and math assessments, in Colombia and similar contexts of forced displacement.

Authors

• Fatima Zahra, New York University; Harvard University

• Hirokazu Yoshikawa, Global TIES for Children, New York University

• Kimberly E Foulds, Sesame Workshop

• Priyamvada Tiwari, NYU Global TIES for Children