Recycle, reuse, [reexamine], reintroduce: leveraging diverse data to build sustainable learning loops

Group Submission Type: Formal Panel Session

Proposal

Program innovations across sectors have raised the quality of life. They have also led to fads and ineffective programming. How should organizations innovate in education, particularly in low-resource environments? How do we raise the bar for years to come, doing the most good for the most people without doing harm? We argue that the answer rests not in invention but iteration. By building sustainable learning loops using diverse forms of data, we can create foundations for positive and lasting change.

Pencils of Promise, Educate!, and Bridge International Academies are three organizations working to improve the quality of education for students across sub-Saharan Africa and Asia. We aim to increase the capacity of teachers and students alike, with an understanding that there is no substitute if we want sustained change. While our approaches are different, we are united in the belief that results require data. This data must examine intermediate drivers of learning and measure student learning gains or later-in-life outcomes. This data must be a monitoring tool, a driving force for internal learning, and a means for evaluation. It must see on-the-ground truth through a variety of lenses and efficiently capture the information required for decision making. .


Combining our experiences in program development and our understanding of the role that different data played in such efforts, we examine the importance of diversity in data in building a system that encourages learning and spurs improvement through iteration.

We hope to convince practitioners the value of such investments and the need to be strategic and deliberate in capturing information; an organization is never too small to build a strong data foundation, nor too large to reexamine its practices.
With the increased focus on program evaluation, we call on donors and policymakers to be just as diligent in requiring a compelling theory of change and measurement along the implementation pipeline; without it we may know whether some things work or not, but not the critical why that informs the entire education community which levers are the most important to pull. In fact, rigorous evaluation may be more prevalent and come at a lower cost if it can rely on a well-established system of administrative data.
We ask academics to help make research partnerships part of the learning strategy within any organization, by considering a) the provision of data early and often to support improvement (and the ethics of not providing such information), and b) an examination of existing processes of survey development to reduce waste.

We conclude our session by looking to our colleagues in the audience, asking them to share their experiences for some of the challenges we have encountered. What has been the most valuable research other organizations have embarked on and why? Such examples will help us interrogate our own systems. We often rely on the sharing of methods and tools; what are the new open-sourced measurements that organizations can adapt? Finally, what do organizations do when different data tell different, irreconcilable stories? What questions should we then ask ourselves and how do we turn moments of such conflict into learning? Ultimately, we want data to generate insights, conversation, and improvements - not existential crises.

Sub Unit

• General Pool

Individual Presentations

• Teacher support and quick iterations: Using a multidimensional approach to both implement and improve programs - KC Kennedy, Pencils of Promise

• Shifting teacher pedagogy: Leveraging a diverse toolbox to assess ongoing program effectiveness - Meghan Mahoney, Educate!

• Making math mentoring work at scale: The development and roll-out of a cross-age tutoring program in Kenya - Lisa F Chen, Bridge International Academies