Measuring children’s social-emotional development in context: Developing and Adapting Tasks for Emotion Recognition and Regulation in the Syrian Response Region

• In Event: Evaluating educational mass media for children affected by conflict and displacement: Lessons learned from Jordan and Colombia

Proposal

Ahlan Simsim is a comprehensive early childhood initiative for the Syrian refugee response region in the Middle East. At its core is the new Sesame Workshop broadcast program, itself titled Ahlan Simsim, which focuses on supporting children’s social and emotional development. A curated selection of 26 episodes across the first four seasons has served as the focus of an impact evaluation conducted in 2022. Child emotion identification and regulation are central to the curricular goals and theory of change of the program. Although these child outcomes have received considerable attention in Western populations (Taylor, et al 2017), and have been linked to important developmental milestones and school success (Durlak, 2011), the validated tools measuring them are scarce in the Levant region.

Building from protocols validated across other countries and regions (i.e. Brenick et al., 2010; Obradovic et al., 2018; Provenzi et al., 2017) we developed tasks to assess children’s abilities to: (i) identify basic and complex emotions, (ii) recognize these sample emotions from facial expressions, (iii) identify their own and other’s emotions based on situational vignettes, and (iv) to identify key regulation strategies when facing such situations. Key local and global consultants and informants provided feedback on the developmental and cultural appropriateness of our stimuli. Local illustrators participated in the creation of engaging stimuli for children.

Methods
We implemented this assessment protocol as part of a classroom-based randomized control trial in the regions of Irbid, Balqa and Karak in Jordan, to evaluate the impact of the Ahlan Simsim show on its intended SEL outcomes. A total of 216 preschools were randomly assigned to treatment and control conditions, with the treatment group receiving daily exposure to a 26-minute episode of Ahlan Simsim over twelve weeks in addition to the Jordan Ministry of Education KG2 curriculum, and the control group receiving only the Ministry of Education curriculum. Each of the 216 classrooms was administered our child assessment tool at endline during May 2022.

We used psychometric indicators and field observations to analyze these data exploring children’s use of specific target words to name central emotions and strategies - using both frequencies and open coding. We evaluate basic reliability and validity indicators exploring agreement among items, coders, and situations. We test whether performance in these new tasks is associated with demographic characteristics of children, their schools and previous exposure to the show.

Our results indicate adequate reliabilities in the final version of our measures, as well as important variation in children’s performance to differential tasks. Taken together, this evidence will be used to experimentally evaluate: (1) intervention impacts and (2) measurement tool development process and performance.

In our presentation, we will discuss this process of measurement development as an alternative to off-the-shelf measurement, highlighting the process of instrument development that closely align to intervention efforts and responding to the particular challenges and affordances of different settings and populations. Lastly, results will be presented in the context of the broader impact evaluation with child assessment and emotional identification modeled in conjunction with impact outcomes.

Authors

• Casey Moran, New York University

• Dennis Hilgendorf, Global TIES for Children, New York University

• Hirokazu Yoshikawa, Global TIES for Children, New York University

• Kate Schwartz, Global TIES for Children, New York University