ENJOY: A digital framework for assessing and promoting education practitioners’ evidence practices

Group Submission Type: Formal Panel Session

Proposal

Education practitioner organizations have designed and implemented a range of innovative, context-driven solutions to education challenges, but can face challenges in generating and using evidence to refine, sustain, resource, and scale their programs. Participants in this panel will come away with an understanding of the Evidence Navigator Journey (ENJOY) tool, a new diagnostic framework for assessing organizational and programmatic evidence practices, and how they can apply the tool in their own organizations to identify their strengths and opportunities to use evidence more effectively.

More specifically, this panel will share the experiences of three organizations collaborating to build, test, validate, and use ENJOY, which was conceived and has been supported by a foundation in the child development and learning space, which will also chair the panel. ENJOY is a digital assessment tool that helps education practitioners and others score their evidence practices (at the level of the organization and for specific programs) and reflect on how to close critical gaps. Presenting authors will share how they supported ENJOY’s development and/or are using ENJOY to advance evidence-informed decision making. They will also share findings on what has worked to date in using ENJOY to diagnose evidence needs, lessons on what short- and long-term supports can help practitioner organizations generate and use quality evidence, and what remains to be done to support their generation and use of evidence.

Specifically:
• The first presenter (from an advisory organization that works with partners to make sense of and act on evidence in education) will share a paper presenting a set of “evidence standards” designed to help organizations build a shared understanding of what “good evidence” looks like. These evidence standards are the conceptual basis for ENJOY and emphasize four domains of evidence-informed practice: (1) Effectiveness – the degree to which an intervention can deliver the intended outcome, (2) Implementability – the degree of ease with which an intervention can be implemented successfully in a given context, (3,4) Transferability and scalability – the ability to transfer an intervention to a new setting (balancing fidelity with adaptation) and the ability to expand the reach of interventions.

• The second presenter (from a research and data analytics consultancy acting as learning partner to the foundation supporting ENJOY) will describe the process used to transform the evidence standards proposed by the first presenter into the ENJOY digital assessment tool. This author will also share early use cases of the tool by the Chair’s organization (which funded and led ENJOY’s development and is currently supporting testing and validation efforts). Early applications of the ENJOY framework include: 1) designing questionnaires measuring the extent of evidence use by policymakers in three countries; 2) evaluating nominations for a prestigious award supporting evidence-based best practice for advancing child learning and education; and 3) screening applicants for a program offering short-term evidence infusion sprints to small- to mid-sized education NGOs, pairing them with researchers in the child development and learning space.

• The third presenter (from a global organization that works to promote the implementation of impactful and scalable education interventions) will share the findings of a pilot (expected in November 2024) focused on helping a cohort of 100 education innovators use ENJOY to reflect and improve on their approaches to measuring impact and preparing to scale. This presenter will describe how innovators used ENJOY, characteristics contributing to variations in use, and innovators’ perspectives on strengths and challenges related to strengthening their evidence practices. The organization will also reflect on its experience building innovators’ capacity to use ENJOY and share ideas for how the organization may use ENJOY in the future.

A discussant (the organization that runs the evidence infusion sprints mentioned above) will react to these presentations drawing on their experience providing education organizations with highly personalized supports and expertise to strengthen the evidence base of their education solution. Their comments will also draw on the findings of a recent case study assessing the short- and longer-term influence of their program on the evidence practices of education NGOs. A second discussant, an NGO participant in the sprints, will share their evidence journey—the challenges they have faced in accessing and using the right evidence to strengthen their work, how they have addressed those challenges, and what needs they continue to grapple with. The NGO participant will also provide initial reactions to the presentation of ENJOY, such as how their organization can use the framework or how the framework can support their evidence journey.

Sub Unit

• Monitoring and Evaluation SIG

Chair

• Donika Dimovska, Jacobs Foundation

Individual Presentations

• Using evidence standards to move beyond a focus on evidence on effectiveness to assess evidence on implementability and transferability - Maryanna Abdo

• Transforming evidence standards into a digital tool for assessing the strength of evidence generation and use - Alejandra Aponte, Mathematica

• Piloting the use of a digital assessment tool to strengthen innovators’ capacity to measure impact and push for scale - Lasse Leponiemi, The HundrED Foundation

Discussants

• Eliza Berg, MIT Solve

• Drew R Edwards, MIT Solve