Establishing the practice of evidence-based policy development in South Africa through large scale RCTs

• In Event: Better “M” for better “E”: What is the price of data ownership?

Proposal

One of the biggest developmental challenges facing South Africa is the high number of children who do not learn to read for meaning in the early years of school. With reading being a fundamental learning skill, this problem has become a major priority for government. There was no systematic local evidence of an intervention that would improve reading results at scale. Huge investments have been made in in-service teacher training programs, largely by donors, but with very little real impact on reading results.

In response the Department of Basic Education (DBE) launched the Early Grade Reading Study I in 2015 in partnership with local research institutions and Georgetown University in the US, co-funded by the private sector. At the core of this RCT was a comparison between two interventions providing a structured learning program with centralized training, and the same program with specialist on-site coaching. After two years of intervention the coaching model produced results that were approximately 40% of a year of learning ahead of the students in the control group. At this point the DBE approached USAID to fund next steps in the Early Grade Reading Study.

USAID/South Africa shared the concern of the DBE that the on-site coaching model would be too expensive to take to scale and suggested a tablet-based intervention with virtual coaching modelled on the successful USAID-funded Tusome program in Kenya. A delegation of government officials and academics visited Kenya with USAID to learn more about the program and subsequently designed the USAID-funded Early Grade Reading Study II (EGRS II) that would compare the cost-effectiveness of on-site coaching with that of virtual coaching.

In the meantime, the DBE convened education experts and stakeholders, including USAID, to interpret the results of the EGRS I and develop an improvement plan for the sector that was subsequently reported to parliament. The improvement plan recommends that all provinces employ coaches in schools where teachers would particularly benefit from ongoing support. It further recommends ongoing research to evaluate the on-site coaching model at scale in a province.

Following the improvement plan USAID is supporting evidence-based policy reform in South Africa by funding the implementation of the structured learning program, combined with on-site coaching, at scale in the North West Province, and has separately contracted an evaluator to conduct data collection and analysis for an RCT evaluation of this intervention, led by the research unit within the DBE. USAID is also continuing to fund the RCT in Mpumalanga Province to establish the cost-effectiveness of a virtual coach compared to an on-site coach.

During the panel, USAID/South Africa and the Research Unit at the South African Department of Basic Education will discuss how the practice of using systematic data collection to inform evidence-based policy development and implementation is being established despite resource and uptake challenges.

Authors

• Stephen Taylor, Department of Basic Education, South Africa

• Carien Vorster, Education Project Management Specialist, USAID/South Africa