BackHome
Refresh: OffView Personal Schedule
Session Submission Summary
Share...

Direct link:

Strengthening holistic learning outcome measurement systems in Colombia and Peru: From frameworks to strategies for coherence and inclusion

Wed, March 13, 8:00 to 9:30am, Hyatt Regency Miami, Floor: Third Level, Johnson 1

Group Submission Type: Formal Panel Session

Proposal

Background
Efforts to gauge and enhance children's holistic learning, encompassing academic, social, and emotional skills crucial for their growth, have surged thanks to the pursuit of Sustainable Development Goal #4. However, these initiatives suffer from a lack of coordination and fragmentation at global, regional, national, and local levels. Often, they narrowly focus on isolated measures or data points, disregarding the broader education systems they are embedded within. This limitation prompted the emergence of frameworks that recognize information and data as critical elements, seeking to address fragmentation and boost effectiveness. (Cambridge Education, 2021; Foster-Fishman, Nowell, & Yang, 2007).

For example, the Research on Improving Systems of Education (RISE) framework aims to support stakeholders to identify the extent to which education systems are aligned towards supporting academic learning (or other purposes, such as access to education) (Pritchett, 2015). It does so by assessing the extent to which goals (such as those enshrined in curricula, frameworks, and standards), information (such as data and measures), supports (such as in-service and pre-service teacher training), motivation and financing are aligned towards the purpose of academic learning within, between and across stakeholders in educational systems. The framework identifies four feedback loops or sets of stakeholder relationships that are defined by exchange and feedback around the five elements (Spivack et al., 2023). In these relationships, one stakeholder - such as the Ministry of Education - holds another stakeholder - such as front-line service providers - accountable for completing a goal. The Ministry of Education provides support and financing to the front-line service providers, while the front-line service providers have intrinsic and extrinsic motivators to support progress to the goal; progress against the goal is assessed by information. While RISE originally identified four such relationships, research suggests additional relationships and complexities are present in contexts of displacement (Tubbs Dolan, 2018).

By looking across, within, and between elements and relationships, one can identify areas of alignment towards learning goals - and areas of misalignment. This is important given emergent research suggesting that it is unpredictable as to whether any given educational intervention designed to improve learning outcomes will have a sustained impact if it is implemented within a system where elements and relationships are not aligned towards learning (Scur et al., 2016). Similarly, any single effort to generate measurement, data or evidence is unlikely to be successful in ultimately supporting learning unless it is aligned with key systems goals, supports, financing, and motivation.

At the same time, both efforts to generate measurement, data, and evidence on holistic learning outcomes and efforts to strengthen systems around measurement, data, and evidence have been critiqued as perpetuating “color-blind and technocratic” approaches to development (Sriprakash et al., 2019; Novelli et al., 2023). Often dominated by researchers in the Minority World, measurement, data, and evidence generation initiatives risk reifying and perpetuating inequities in who is accepted as a knowledge producer, grounded in legacies of colonialism and racism. Systems-level frameworks, meanwhile, fail to acknowledge discrimination, a social process that is embedded in stakeholder feedback loops (Sriprakash et al., 2019).

The Panel
In this panel, we present an effort by a team of Colombian, Peruvian, American, Argentinian, Chilean, Sri Lankan, and Dominican-American researchers to support stakeholders in Colombia and Peru to strengthen the measurement of holistic learning outcomes and the systems in which the resulting information is generated and used in a way that promotes fairness, equity, and inclusion of historically marginalized groups. In terms of our own positionality in this work, many of us have previously participated in or led national or regional initiatives to measure and generate data and evidence on children’s holistic learning. Our critical reflections on these initiatives have spurred us to design such initiatives with a systems lens and with an aim of interrogating power and privilege dynamics.

In Paper 1, we describe the iterative process of developing a framework for the analysis of holistic learning outcome measurement systems. The framework development process sought to integrate and extend two existing frameworks for education systems analysis in two ways: (1) to encompass social and emotional learning goals as well as academic learning goals; and (2) to ensure fairness and equity in the generation and use of information for marginalized populations, including displaced populations; Black, indigenous, and ethnic minority populations; LGBTQ+ persons; and children with disabilities. This framework then served as the basis for the development of a mixed-methods study that sought to identify the types of information on holistic learning outcomes generated, used, and shared by national and sub-national policymakers, researchers, NGO staff, and teachers; as well as to identify areas of alignment and misalignment within and between these stakeholders in information, goals, and supports. The history, context for, and results of these studies in Colombia and Peru will be presented in Papers 2 and 3, respectively. Finally, in Paper 4 we present our collaborative process for developing strategies to strengthen holistic learning outcome measurement systems based on the results of the mixed-methods studies. This process involved triangulating interpretation of the data and experiences of the research team and a Steering Committee in each country.

Discussion
As efforts to strengthen measurement, data and evidence systems globally proliferate (KIX, 2023), this panel aims to increase awareness of a conceptual framework and a set of open-access tools that can support the cohesiveness and equity of such efforts. We also aim to showcase a collaborative, non-linear process to systems strengthening that can potentially serve as a model for other such efforts.

Sub Unit

Chair

Individual Presentations

Discussant