Reducing Attrition in Longitudinal Tracking of Pupils

• In Event: Poster session III
  In Poster Session: Poster session 3

Proposal

Missing data is common in almost all research and is especially a challenge in longitudinal studies. A review of longitudinal studies in public health found attrition rates from 30 to 70% are often reported1, and the What Works Clearinghouse considers overall attrition of 55% as a tolerable threat of bias2. While there are several statistical methods to deal with bias and decreased power of findings, a sustainable monitoring system needs a strategy to prevent as much missing data as possible. This poster will present strategies used by the DfID/UNICEF Reading and Numeracy Activity (RANA) to monitor teacher performance and pupil learning outcomes.
For RANA’s monitoring and evaluation (M&E) purposes, we covered 199 schools (80 IQS and 119 public schools), where enumerators interview the head teacher, lead reading teacher, and one classroom teacher from each of primary grades 1 through 3. Additionally, one classroom from the same teacher is observed using a Stallings-type tool and pupils are selected at random from that classroom. Because the primary M&E indicators for the intervention involved P2 pupil reading performance, we sampled 10 pupils per P2 classroom and 3 pupils from P1 and P3 who were to be tracked from the beginning of the year to end of the same school year. This yielded a final sample of 589 teachers, 199 head and lead teachers, and 3,425 pupils in 2016-17 and 4,224 pupils in 2017-18.
We were able to reach all of our targeted teachers, lead teachers and head teachers by electronically monitoring the performance of government School Support Officers. Each interview was verified by recording the teacher’s signature along with the GPS coordinates of the school. Each submission automatically captured the serial number of the device used to record the data, and the name and phone number of the enumerator. Enumerator supervisors in country then had access to this data through an automated dashboard to track school coverage.
We achieve longitudinal data at the pupil level within-year, rather than between-years as pupil mobility is likely high and tracking pupils over a longer duration of time is challenging in most development contexts. For instance, attrition rate in a 2014 impact evaluation conducted by Bridge International Academies of a similar early grade reading and math intervention in Kenya yielded an attrition rate of 41.1% between 2013 and 2014. In RANA’s first full year of implementation, we had a similar attrition rate of 42% in 2016-17, and reduced the attrition to 19.8% in 2017-18.
The marked decrease in attrition is attributed to an adjustment made to the method used when tracking participating children. The original method included providing each enumerator with a list of school assignments along with the list of names of the pupils (and the grades they are attending) who were chosen to participate in the beginning of each academic year. In 2017-18, the FHI 360 data capture process was enhanced to include a headshot photograph of the participating pupil. This adjustment approximately halved the attrition rate from the previous year from 42.3% to 19.8%.


1Gustavson, K., Soest, T. V., Karevold, E., & Røysamb, E. (2012). Attrition and generalizability in longitudinal studies: Findings from a 15-year population-based study and a Monte Carlo simulation study. BMC Public Health, 12(1). doi:10.1186/1471-2458-12-918
2What Works Clearinghouse. (2017). Standards Handbook. Version 4.0. Princeton, NJ: What Works Clearinghouse. (ERIC Document Reproduction Service No. ED 577 036)

Authors

• Wael Moussa, FHI 360

• Brian D. Dooley, FHI 360