Evaluating learning outcomes in fragile contexts

• In Event: Education Planning and Policy in Fragile Contexts

Proposal

When evaluating education programming, understanding ‘pockets of resistance’ and hidden issues is a key step in making informed investments to address inequities in access, learning outcomes and retention. This is particularly important when working in fragile and conflict-affected contexts, where the scarcity of data leads to programming being often informed by small-scale, non-representative studies and by assumptions about marginalized sub-groups. Despite the glaring need for in-depth analysis, the majority of the evaluation processes still tend to use narrow approaches, restricting data analysis to tracking learning scores and using single-method research and analysis methodologies. This paper will discuss how the investment in complex, multi-layered triangulation to explore and interpret data may contribute to a better understanding of how multiple barriers intersect to affect learning, particularly for the most marginalized.


This paper will reflect on an organizational experience of analyzing complex datasets emerging from adolescent-focused education programming in eight countries, including fragile and conflict-affected contexts, during the last five years. The use of extensive triangulation processes in girls’ education programming led to the identification of ‘pockets of resistance’, or sub-groups of students who lag behind in performance, but also face a conjunction of other factors that affect their capacity to fully engage in learning processes. Multi-layered analysis of different data sources allowed for the identification of ‘hidden issues’– factors that affect performance and retention, but are generally ‘hidden’ in survey responses, such as gender-based violence, patterns of irregular attendance and migration, practices that limit engagement in class, and gendered behavior. Multi-dimensional triangulated analysis is also applied to learning assessments, with the analysis of tests tailored to (a) identify specific areas of weak performance that may indicate the need for reinforcement of teaching skills, as well as (b) identify skill levels in multiple languages, particularly when the language of instruction is not the mother tongue and (c) cross-reference literacy and numeracy results with the acquisition of life skills that enhance classroom participation.


The use of well-designed and analyzed evaluation studies to explore the contextual barriers and opportunities for education programming in fragile contexts, identifying and unpacking nuanced patterns of exclusion, generates valuable data to inform practitioners to refine programming on the ground, as well as to support policy development and allocation of development resources. This approach allows programs to engage in iterative processes of tailoring and refining implementation strategies to address specific areas of inequity, maximizing the impact of education programming among marginalised sub-groups.

Authors

• Lotte Marianne Pires Renault, CARE USA

• Amanda Moll, CARE USA