Impact Evaluation of a School Leadership Development Programme in Western Cape, South Africa

• In Event: Leadership, Digital Transformation, and Critical Media Literacy: Evaluating Educational Innovations Across Contexts

Proposal

Effective school management has consistently been linked to improved student learning outcomes (e.g., Bloom et al., 2015). In low- and middle-income countries (LMICs), both school management quality and student learning outcomes are generally poor (Azevedo et al., 2022; Bloom et al., 2015). Improving the productivity of key personnel in school systems, especially school leaders, seems like a promising strategy for enhancing student learning. This model is particularly appealing because it implies that by training a few key personnel, school-wide impacts can be achieved, making it a cost-effective solution. However, the evidence on the effectiveness of leadership training programs in LMICs and their impact on students is mixed. For instance, a meta-analysis by Anand et al. (2023) of 20 studies found that the effects of educational management interventions on student learning are minor (0.03 - 0.04 SD). The study concluded that the primary barriers to program effectiveness are the lack of incentives to implement recommendations and the long, indirect causal link between improved management practices and student performance.

In this study, we evaluated the Instructional Leadership Development Programme (ILDP) in Western Cape, South Africa, using a mixed-method approach. This included a randomized control trial with 79 schools (39 treatment, 40 control), a process evaluation measuring compliance, and a qualitative component to understand program experiences. The ILDP aims to enhance teaching and learning quality through training, coaching, and mentoring school leaders. Its tailored approach is designed to meet the specific needs of school leaders. The two-year program consists of group training for leaders covering school management and effective teaching techniques, along with weekly virtual and in-person individual coaching sessions. The program’s Theory of Change suggests that training school leaders will result in school-wide improvements in teaching quality and student performance.
Findings
Due to high time commitments required from school leaders, compliance rates with the ILDP were low, with 56% of treatment schools completing the program and only 33% regularly attending activities. Lower-resourced schools were more likely to complete the programme, possibly viewing it as particularly beneficial. Factors affecting low completion rates included financial constraints, personal reasons, lack of promotion, and insufficient support from the school or supervisors.
For schools that completed the program, some ILDP practices were implemented. Specifically, ILDP-compliant schools were more likely to adopt best practices in strategic planning, monitor changes, personalize instructional plans, and provide direct feedback to teachers. However, improvements in school management practices were modest between treatment and control schools. The average treatment effect (ATE) showed a 5% difference in management practices index scores between the two groups (difference = 0.155, p-value = 0.103). Adjusting for non-universal participation, the treatment-on-the-treated (TOT) estimate indicated a 9.3% change compared to the control group (difference = 0.279, p-value = 0.068). Both estimates offer valuable insights: the ATE is useful for comparing ILDP impacts with other programs, while the TOT measures the impact on schools willing and able to participate in an intensive leadership development program.
We found no significant impact on teaching quality or student test scores.
This lack of impact on downstream outcomes is likely due to several factors. First, the sample size may have been too small to detect changes in student test scores. Second, more time might be needed for the changes to fully take effect due to the program’s long causal chain and gradual implementation. Qualitative findings indicated some positive changes in school culture—such as improved student discipline, classroom management, and teaching quality—but these were often confined to specific grades or phases and lacked consistency, possibly preventing school-wide effects. Improvements may have occurred in classes that consistently implemented the new techniques.


Anand, Gautam, Aishwarya Atluri, Lee Crawfurd, Todd Pugatch, and Ketki Sheth. "Improving school management in low and middle income countries: A systematic review." Economics of Education Review 97 (2023): 102464.

Azevedo, Joao Pedro, Halsey Rogers, Ellinore Ahlgren, Maryam Akmal, Marie-Helene Cloutier, Elaine Ding, Ahmed Raza et al. "The state of global learning poverty: 2022 update." World Bank et al. (2022).
Bloom, Nicholas, Renata Lemos, Raffaella Sadun, and John Van Reenen. "Does management matter in schools?." The Economic Journal 125, no. 584 (2015): 647-674.

Authors

• valentina brailovskaya, IDinsight

• Rico Bergemann, IDinsight