Long-term follow-up studies: potential, challenges, and evidence from Haiti and Burkina Faso

• In Event: Using long-term follow-up to learn more about projects’ impacts: Evidence from Honduras, Haiti, Burkina Faso, and globally

Proposal

Evaluations are typically restricted to the life cycle of a program or activity, due to constraints such as funding or timing. Yet, theories of change almost always hypothesize long-term impact that goes unmeasured. In this presentation, we discuss the potential added value of long-term follow-up evaluations, challenges that arise with long-term follow-ups, and offer potential strategies to address those challenges.

In this presentation, we will also provide examples of two long-term follow-up evaluations in Haiti and Burkina Faso. In Haiti, the Read to Learn intervention employed a scripted mother-tongue curriculum for 1st and 2nd grade students in two districts of Haiti. An RCT studying the impact of the intervention found statistically significant impact on emergent reading skills such as phonemic awareness and oral reading fluency (Author, et al. 2020). The follow-up evaluation re-tested the same students from treatment and control schools at 4th grade, two years after program completion. Results from this study indicate lasting effects on phonemic awareness, but limited effects on higher level reading skills. Additionally, the follow-up study faced high levels of attrition which may threaten the validity of results. This study offers an example of the challenges of following the same students over time; lessons learned will be presented.

In Burkina Faso, the BRIGHT program included the construction of new schools and provision of complementary services in 132 communities. The evaluation conducted a 10-year follow-up that assessed impacts on enrollment, educational attainment, child labor, and child marriage. The evaluation reported on outcomes relevant for both children (e.g. school enrollment) and young adults (e.g. child marriage, educational attainment). Rather than track down individuals who were school age at the time of the intervention, the study gathered data from all households in BRIGHT villages and comparison villages. The advantage of this approach is that it allows the study to report on population level impacts on BRIGHT villages and avoids the challenge of tracking individuals after many years. The authors found evidence of long-term impacts. Using a regression discontinuity design to estimate causal impacts of the BRIGHT program, the authors found that BRIGHT villages had significantly higher rates of school enrollment and primary school completion and higher test scores. They also found significant reductions in child marriage and employment for female young adults aged 13-22 and reductions in employment for male young adults of the same age range (but no impact on marriage).

Author

• Danice Guzman, University of Notre Dame-Pulte Institute