Session Submission Summary
Share...

Direct link:

Innovating assessment: devising a holistic assessment for education cannot wait in Syria

Tue, April 16, 10:00 to 11:30am, Hyatt Regency, Floor: Atrium (Level 2), Boardroom B

Group Submission Type: Formal Panel Session

Proposal

Education is a fundamental component of humanitarian assistance, contributing to child survival, to safeguarding a sense of normalcy and continuity, and to furthering a sense of dignity and hope for children and their families. The Education Cannot Wait investment for Syria acknowledged that in a complex and protracted crisis, ensuring access to quality education for children requires interventions that consider context-specific social, political and cultural dimensions, NGO and ministry staff capacity to plan, coordinate and implement these interventions, and data on learning to inform them. This panel will focus on the final element of this work: the holistic measurement of learning outcomes and the process of tool development for this context. This topic resonates with CIES members who measure learning outcomes in the context of national and international investments and who work in emergency settings. As more and more measurement work is being done in such contexts, a process for careful contextualization is needed.
While measuring learning outcomes is an important part of the accountability agenda (do we deliver – as teacher, as school, as system – upon our collective promise to provide all children with quality education), this effort focused upon doing so as part of a feedback loop so teachers and systems can improve teaching and learning policies and practices. Further, this initiative prioritizes a view of foundational learning skills broader than reading and math, including social and emotional skills as well. This is particularly important in emergency settings.
The initiative addresses the need for a holistic, contextualized tool for use in Syrian schools. The short-term focus is on developing a tool for quickly obtaining a better understanding than previously available of the level of foundational learning skills students have in early grades in this emergency setting. In this context, students’ exposure and opportunity to attend school and learn skills has varied and teaching to their level is more complex than in other more homogeneous settings. In the first phase of this effort, partners devised and tested a tool to rigorously but briefly capture these skills. In the longer-run, this effort will develop and test guidance for teachers for using the tools in the classroom to adjust instruction.
The initiative has three main partners – Save the Children, UNICEF and NYU Global TIES for Children – and four main activities. First, the partners reviewed early grade learning measurement tools already used in the context of conflict and crisis in Syria to measure students’ social emotional, reading and math skills. They gathered these as well as the datasets for re-analysis aiming to interrogate the psychometric rigor of the tools. Second, the partners built on this work by engaging 17 regional experts in a four-day workshop to add their own expertise in assessment of such skills in Arabic and to systematically review the tools and the results of the psychometric analyses. This workshop group considered which items were functioning well, which should be set aside, and which could be adjusted for a more robust measurement of reading, math and social emotional development. The end product of that second activity fed directly into the third and fourth activities: pilot testing a new and improved tool, conducting new psychometric analyses and feeding the results back to the team of regional experts for consideration and finalization. The final tool provides a snapshot of children’s reading, math and social emotional skills.
This approach to building a contextualized learning assessment is innovative in at least four ways. First, it moved beyond prior efforts to assess learning in the region that had been based on template tools and guidance not created specifically for Arabic. Second, it drew upon a team of Arabic-speaking experts in assessment to ensure appropriate coverage of unique elements of assessment in Modern Standard Arabic when children speak a local dialect. Third, it aimed to combine more learning domains in a similar amount of time used for a single early grade assessment – encompassing reading, math and social emotional domains. Finally, when it pilot tested the new holistic tool, it included psychometric analyses of the new tool to inform not just the status of children’s learning, but also the validity and reliability of the tool in measuring it.
This panel will share the process and results of this initiative in three presentations. First, Dr. Reem Khamis-Dakwar of Adelphi University will share several key principles of Arabic language processing and reading that informed this work. Second, Dr. Carly Tubbs-Dolan and Roxane Caires of NYU Global TIES will present the psychometric properties that drove decisions about what items should be in the new pilot and some of the challenges faced in taking this approach. Finally, Clay Westrope of Save the Children will present the tool that was piloted, the results of that pilot, including reflections on successes and challenges, and then with an assist from Dr. Tubbs-Dolan, share the new tool’s psychometrics. After a brief intervention by the Discussant, Manuel Cardoso of UNICEF, we look forward to a discussion of the pros and cons of our approach and to learning what the CIES members who attend think that we should do next.

Sub Unit

Chair

Individual Presentations

Discussant