Innovative and contextually relevant measurement instruments to enhance Social and Emotional Learning (SEL) monitoring and evaluation processes

Group Submission Type: Formal Panel Session

Proposal

Children’s social and emotional learning (SEL) has gained increasing interest in emergencies, given its advantages to help children deal with the adverse physiological and psychological effects of being exposed to the risk factors that come with living in vulnerable contexts (Baker-Henningham et al., 2009; Jordans et al., 2010; Puerta et al., 2016; Torrente et al., 2019). This growing interest in SEL has boosted a variety of educational programs aimed at developing children’s social and emotional skills in the school context (Torrente et al., 2015). To conduct proper monitoring and evaluation processes for the implementation of SEL programs, there is a need to develop valid, reliable and context-responsive assessment tools of children’s SEL skills. To date, important progress has been made in the design of a wide variety of procedures and approaches to measure SEL skills (Denham, 2015; Brownell et al., 2015; INEE & EASEL, 2020). These methods vary across several methodological aspects, for example, who is providing the information (eg., children, peers, parents, teachers), what is the intended use (eg., screening, formative, summative), and how is the information collected (eg., ratings, observations, direct assessment, interviews), among others.

In spite of the progress of the field, there are still some challenges and limitations to be addressed. For example, research should be looking at overcoming social desirability biases as well as developing more direct performance-based assessments of SEL skills, while optimizing feasibility and cost-effectiveness of the instruments administration procedures (Denham, 2015; McKnown, 2015). Performance-based assessments test children directly by asking them to demonstrate their skill by completing a set of tasks (INEE & EASEL, 2020). These types of assessments are usually conducted by a trained examiner through individual verbal interviews where children are asked to solve social problems or to identify themselves with a set of alternative descriptions. Although there are several performance-based tools already developed and validated across different populations (eg., IDELA, ISELA, HALDO, PSRA see INEE & EASEL, 2020), these instruments rely heavily on the qualification of the examiner, which makes the data collection procedure very expensive for large-scale studies. With the development of digital technologies, one potential contribution for this field lies in the development of media-based digital platforms that increase children’s capacity to respond to the instruments without the need of one-to-one interviews.

Another important challenge is that the way SEL skills are defined, codified, and operationalized in research and how these conceptualizations are translated in programs and practice are not always consistent or agreed upon. This trend is partly because SEL skills are highly socially normative and dependent on the environment within which they are expressed (Jones et al., 2016). The context-specificity of SEL skills affects how they are measured as well. In the absence of a more nuanced understanding of how SEL skills manifest in a specific context, some measures may be mischaracterized or biased. A majority of the measures that have been developed for SEL were initially developed for use in high-resource, stable contexts (Halle & Darling-Churchill, 2016); the measures that are regularly used in low-resource and fragile contexts often do not sufficiently capture the dimensions of identified SEL skills (Jukes et al., 2018; Lee et al., 2019). A recent review (INEE & EASEL, 2020) identified and mapped existing SEL assessment tools used in the international education in emergencies sector. One of the main findings from this study was the need to develop more culturally relevant measurement tools. They stress that priorities for SEL measurement vary by context and that the local communities are the ones who should decide on their own social and emotional needs and expressions.

In sum, important progress has been made in SEL monitoring and evaluation processes. Nevertheless, there is still important work to be done to advance in the development of SEL measurement tools that directly assess children’s skills in a culturally responsive manner. To address these issues, this panel presents four papers aimed at contributing to this goal.The first paper discusses the process of developing an app-based version of the Berkeley Puppet Interview, focusing on the learned lessons from developing and administering SEL surveys using this platform in Liberia, Honduras, and Haiti. The second paper presents an experience of a systematic qualitative study that aims to understand how local communities in Honduras (eg., children, parents and teachers, policy makers, stakeholders) define and prioritize student social and emotional learning (SEL), and how this information can be used to produce contextually relevant assessment instruments in Honduras. The third paper presents the process of developing a game-based mobile app aimed at measuring SEL skills of preschool and early primary school children from Colombia, discussing the learned lessons from designing and piloting this tool with young children. Through a survey, market analysis, and interviews, the final paper provides an overview of the challenges and enablers of implementing game-based assessments (as described in the first three papers) in emergency contexts. The integrated discussion of these four papers make a significant contribution to the current methodological, financial, and cultural limitations of the SEL instruments developed to date in emergencies.

Sub Unit

• Monitoring and Evaluation SIG

Chair

• Nikhit D'Sa, University of Notre Dame

Individual Presentations

• Adapting and administering the Berkeley Puppet Interview digitally in low-resource and crisis contexts - Aimee Lyons, University of Notre Dame-Pulte Institute; Nikhit D'Sa, University of Notre Dame

• Developing contextually-relevant SEL measures in Honduras: Exploring new ways in which to measure skills for children - Camilo Vieira, Universidad del Norte; German Edgardo Moncada Godoy, UPNFM

• Game-based assessment of socioemotional skills of preschool and early primary school students: Design and piloting of a tablet app - Enrique Chaux, Universidad de los Andes; Ana M. Velasquez, Universidad de los Andes

• If you build it, will they come? Enablers and constraints of the adoption of a gamified holistic learning application - Lindsay Brown, NYU Global TIES; Kahina Boulegroun, War Child Holland; Roxane Caires, New York University; Abiraahmi Shankar, NYU Global TIES for Children