Institutionalizing Evidence-Based Decision-Making: Embedding a Culture of Evidence Use

Group Submission Type: Formal Panel Session

Proposal

Institutionalizing Evidence-Based Decision-Making: Embedding a Culture of Evidence Use

Despite increased enrollments beyond early childhood education, an estimated 250 million children cannot read, write or count well, whether they have been to school or not. The pandemic has exacerbated this learning crisis: the closure of schools and universities during the COVID-19 pandemic has affected more than 1.5 billion children and youth worldwide. Despite many governments employing distance learning strategies, an OECD study across 59 countries showed that only about half of the students were able to access all or most of the curriculum, with vast disparities across and within countries. It is estimated that the learning loss that has occurred during the pandemic is equivalent to a loss of between 7 and 10 percent of lifetime income.

While policy makers widely recognise the fact that education serves as an engine for economic growth through the accumulation of human capital, education is also strongly associated with boosting levels of social capital. Designing and implementing evidence-based policies and programs can better support children’s holistic development and learning, reduce learning disparities across the globe, and bring about more just and inclusive educational futures. The development of social capital through formal education has a causal relationship to multiple forms of civic and social engagement, including voter turnout, tolerance and inclusion, and the acquisition of political knowledge, which leads to developing more just and equitable societies.

This panel responds to the need to work with governments to improve their ability to collect and utilize data to develop evidence-informed priorities, programs, and policies. A research and policy NGO is addressing the need for evidence-based decision-making through our embedded labs, which are teams within government organizations working to strengthen the use of data and evidence in public policy. When designing evidence labs, a starting point is to consider a government partner’s policy priorities and research questions, what data is available for decision-making, and how the government is currently generating and using data and evidence. Based on this analysis, we collaborate with a government partner to create a work plan for the lab.

Through partnerships with governments in Côte d’Ivoire, Peru, and Kenya, we have developed tools including a needs assessment, a data and evidence use assessment, and a data mapping toolkit to determine a partner’s current state of data and evidence generation and use. This panel will discuss our experiences developing and using these tools.

Using a needs assessment tool to identifying policymaker interests, priorities, and needs

Across many of our policy engagements and labs in East Africa, we are using needs assessments where we conduct semi-structured interviews with relevant stakeholders to understand their policy agendas and questions, and where evidence and data may have the most added value. In our education related needs assessments we are interviewing stakeholders both in and outside government to gather insight related to the priorities, policy processes, and data and evidence needs of organizations such as Ministries of Education.

In Kenya, our use of our needs assessment yielded several key findings, including identifying two priority policies. Based on the identification of these two policies and their prioritization among other policy development processes, we tailored our strategy to identify where to focus our policy engagement efforts and developed ‘rapid’ reviews of existing evidence to inform their development.
Upon the development and strategic sharing of these reviews in policy formulation processes informed by our needs assessment, we are working together to develop evidence-based policies. During this presentation, we will present the methodology used to conduct the needs assessment, share more detailed results, and discuss how we are using these results to inform our policy work in Kenya.
Conducting a data and evidence use assessment with the Ministry of Education and Literacy in Côte d’Ivoire

In Côte d’Ivoire, we are co-creating a lab with the Ministry of Education and Literacy (MENA) to use data and rigorous research to design, test, and scale evidence-based programs and policies to improve learning for students.

We have conducted a data and evidence use assessment to improve our understanding of data capacity within MENA, as well as around the culture of research, data, and evidence use. Our methodology consisted of quantitative surveys, qualitative interviews, and a document review using existing resources such as data mapping that had been conducted by education sector partners. Results from our assessment note that there is a gap between policy needs and data and evidence use across the Ministry. We also found poorly documented and standardized procedures for data generation, an average level of connection between databases, and limited resources allocated to MEL teams. However, we did find that there is an appetite among decision-makers in all central structures to access and use these data. During this presentation, we will present the methodology and tools used to conduct the assessment, share more detailed results, and discuss how we are using these results to determine which activities the lab should carry out and what our goals are for the lab’s progress.

Conducting data mapping with the Ministry of Health in Peru

In Peru, we have worked with the Ministry of Health to conduct a data mapping exercise, which involves creating a data directory of existing databases, sources, and processes, among other tools, to unlock data for research purposes, operational decision-making, and organizational learning. To conduct this data mapping, we held interviews with key data owners/users, conducted a desk review to identify key databases and variables, mapped data stakeholders, and reviewed datasets in order to understand which data exist, where they are produced, stored, and shared, and how they are used. Based on the results, we were able to identify research questions that could be answered with the Ministry’s existing data that can inform their policy and programming.

Based on this experience, we have created a data mapping toolkit. During this presentation, we will present the methodology and tools we used to conduct the data mapping, share results, and discuss how the results were used to inform the lab’s data generation and use activities.

Sub Unit

• Monitoring and Evaluation SIG

Organizer

• Shahana Hirji, Innovations for Poverty Action

Chair

• Juan Manuel Hernandez Agramonte Caballero, Innovations for Poverty Action

Individual Presentations

• Using a needs assessment tool to identifying policymaker interests, priorities, and needs - Bethany Park, Innovations For Poverty Action

• Conducting a data and evidence use assessment with the Ministry of Education and Literacy in Côte d’Ivoire - Shahana Hirji, Innovations for Poverty Action

• Conducting data mapping with the Ministry of Health in Peru - Karla Petersen O'Farrill, Innovations For Poverty Action

Discussant

• Vianney Mbonigaba, Innovations for Poverty Action