Spurring systems change to improve equity in education: Measuring the change process

Group Submission Type: Formal Panel Session

Proposal

While donors, NGOs, community-based organizations, and governments continue to press for education reform and equity in education, the breadth and depth of change remain elusive. Programs make changes along the margins but are failing to make transformational changes in the lives of vulnerable and marginalized populations. The international community is starting to look to “systems change” as a way of addressing the root causes of inequality in education. But what is “systems change”? How do we design projects that engage in a true institutional transformation that leads to sustained progress in educational attainment, quality, and the reduction of inequities? How do we measure and monitor “systems change?”
Systems have many definitions. Meadows (2008) defines systems as “an interconnected set of elements that is coherently organized in a way that achieves something (function or purpose).” Holland (1998) defines it as “a configuration of interaction, interdependent parts that are connected through a web of relationships, forming a whole that is greater than the sum of its parts.” Others such as Barbasi (2002) and Gates, Walton & Vidueira (2021) focus on systems as “a set of interrelated elements that interact to achieve a purpose or systems that are nested and networked within each other.” Regardless of the definition, systems are complex and interconnected, and any changes to one component of the system lead to a series of additional intended or unintended changes that ripple through the system. Systems change is an intentional process designed to alter a system by shifting its function or structure through purposeful interventions across the system. Systems change aims to bring about lasting change by altering underlying conditions which make the system operate in a particular way – and change in these systems can be difficult to create and even harder to measure.
Systems change is difficult to measure for a number of reasons. First, systems are complex – like raising a child. Each child is an individual with their own personality. They interact with different environments, teachers, and friends as they grow compared to siblings, so while a “parenting formula” may help, it likely has limited applicability. Expertise in being a parent can help, but one can never eliminate the uncertainty of an outcome. It is the same for systems. Systems change occurs when many changes come together over time, interacting with and building on each of those changes to help the system change. These changes can be difficult to measure both within and across different systems. First, where do you draw boundaries to measure change? People and organizations have different perspectives on where boundaries of the system are, so while boundaries are needed, they are often arbitrary making it difficult to fully account for what needs to be measured. Second, any attempt to measure systems change requires people to fully understand the cause-and-effect relationships. These relationships can take time to account for and may be hard to measure. Finally, measuring systems change take time – often years, something donor programs may or may not be willing to do.
This panel brings together a series of evaluations focused on understanding institutional systems reform and how to measure and monitor change. The first presentation entitled Measuring Systems Change: Improving access and equity to middle school education presents the findings from a longitudinal political economy analysis. The presentation focuses on how targeted donor interventions created new systems to improve teacher recruitment and data use for improved decision-making around budgeting middle school delivery. It shows how complex systems and a complicated network of actors reduced the sustainability of systems change interventions. The results also emphasize the need for using systems thinking when developing institutional strengthening interventions intended to improve quality and inequity in education. The second presentation, entitled Improving the quality and relevance of professional education and training focuses on an intervention aimed at strengthening the TVET governance system and improving the alignment of training with the needs of the private sector. The presentation will show how some changes were not feasible and how stakeholders came together to solve the complex problem by increasing the capacity of the private sector to identify technical skills needed and building stronger relationships among the private sector and education actors. The third presentation focuses on measurement and how to monitor and track systems change. Developing an early childhood development (ECD) systems measurement framework, this presentation draws on work with the Hilton Foundation across Mozambique, Tanzania and Kenya. Th presentation highlights how the team has empowered local researchers and organizations to develop measures to monitor Early childhood education in each country, monitor and assess systems change, how to measure it, and how to think about the measurement systems over time.

Sub Unit

• General Pool

Chair

• Ivonne Padilla, Mathematica

Individual Presentations

• Developing an early childhood development (ECD) systems measurement framework - Laura Meyer, Mathematica

• Improving the quality and relevance of vocational education and training - Ivonne Padilla, Mathematica