Gathering community perspectives through implementation research: Understanding variation, fears, and expectations to aid a nation-wide edtech scale-up in Malawi

• In Event: Enhancing Development Projects across Cultures

Proposal

In recent years, education technology (EdTech) programs have become increasingly prevalent in Sub-Saharan Africa and have been shown to be one of the most effective education interventions in developing contexts (McEwan 2015; Damon et al., 2016; Conn 2017; Rodriguez-Segura 2020). Rigorous research conducted in Africa over the past decade has shown the positive effects of onebillion’s onecourse, an award-winning tablet-based curriculum, on children’s literacy and numeracy development in both in-school and out-of-school settings (Pitchford et al., 2017; King et al., 2019; Levesque et al., 2020; Levesque et al., 2022). A 15-month randomized controlled trial (RCT) of 5 EdTech softwares including onecourse, conducted for the Global Learning XPrize with out-of-school children in Tanzania, produced effect sizes of .46-.59 in literacy and .56-1.09 in numeracy (King et al., 2019). Further, two RCTs of onecourse with Malawian Standard 2 learners in government primary schools produced effect sizes of .34-.37 in literacy and .29-.54 in numeracy after 8 months and 13 months (Levesque et al., 2020; Levesque et al., 2022).

In 2022, the Building Education Foundations through Innovation & Technology (BEFIT) program in Malawi was launched by the Government of Malawi (GOM) to use onecourse as supplemental learning nationwide. The vision is to empower every primary school child in Malawi with the foundational literacy and numeracy skills they need to realize their full potential. BEFIT will be rolled out over the next 6 years starting in September 2023 to all 5,770 public primary schools, ultimately serving 3.5 million learners in Standards 1-4 annually. Children will be provided daily access to onecourse software in addition to regular classes. The software is in Chichewa and will be aligned with the national curriculum. Each tablet will be used by an average of five learners per day during school hours. Scaling this program country-wide is anticipated to significantly increase foundational literacy and numeracy skills for students.

While onecourse interventions have produced positive effects in both literacy and numeracy, careful monitoring is needed to ensure sustained effects upon scaling. Without proper monitoring of implementation, the quality of delivery and uptake may be compromised at such a large scale. Many programs produce reduced effects at scale, in part because implementation factors are not carefully considered (Stern et al., 2021). In an effort to ensure the continued positive effects on children’s literacy and numeracy development as BEFIT scales, we are conducting Implementation Research (IR) during the first years of the program rollout. IR can be defined as the study of methods to improve the uptake, implementation, and translation of research findings into routine and common practices, obtaining information to guide scale-up and sustainability (MEASURE Evaluation Implementation Research Technical Working Group, 2015). Tanzania offers an example of successful use of IR related to a tech-supported, teacher continuous professional development program. The program launched in 2022 and used design-based IR to evaluate and iterate during the first year. At the end of the first year, 70% of teachers said the program had improved their teaching practice and 73% agreed that students progressed in their learning as a result (Koomat et al., 2022).

This paper introduces our plan for IR during the BEFIT roll-out, lessons learned during the current planning year, and insights into community perspectives of the onecourse program in Malawi. We and our country-based research partners will conduct IR over the course of the next three years in an effort to identify barriers to and enablers of quality implementation as the program scales. IR will involve interviews/observations with key stakeholders at various education levels and in implementation districts spanning Malawi. Specifically, we are targeting stakeholders at the ministry, district, zone, school, and community level to gather information on their understanding of the tablet program and potential barriers to/enablers of program adoption. We seek to better understand contextual differences across the country, anticipate challenges to address during sensitizations and trainings, adapt practices according to community needs, and plan to share our findings with key stakeholders and program implementers.

We recently completed Year 1 (planning year) interviews/observations. Thematic analysis of the qualitative data has revealed key insights into community perspectives, differences between schools, and initial fears of and hopes for the program. During initial interviews, various community and school members communicated a feeling of gratitude for the interview process, noting that those implementing past interventions typically did not ask how they felt about the process and/or program. Follow-up interviews at four initial pilot sites spanning various districts revealed community fears ranging from a fear of the tablet replacing teachers to the technology representing a satanic force. As a result of the interview process, our partners on the ground were able to alleviate these fears through community sensitizations and teacher trainings and eventually secured the buy-in of the parents and teachers. Further, observations at the four initial pilot sites revealed differences across sites in class sizes, associated scheduling challenges, time-on-task decisions, teacher roles, and tablet rotation practices. The information gathered has given us a broad view of how schools are beginning to implement the program. This will help us relay information between schools on what works and what doesn’t while also allowing for typical variation to occur. Further analysis will be conducted and presented at the 2024 CIES conference to understand themes emerging within and between districts and schools. The data gathered during the planning year is helping to inform the rollout of BEFIT Cohort 1 in an effort to ensure sustained positive outcomes for learners and communities as the program scales.

Authors

• Celeste Lopez, Imagine Worlwide

• Symon Winiko, University of Malawi

• Karen Levesque, Imagine Worldwide