Investigating Malawi's reading gap

• In Event: Disentangling why children cannot read: Introducing USAID’s new oral language module

Proposal

Malawi has one of the lowest levels of education in the world. It ranked 171st out of 189 countries listed on the United Nations Development Programme’s 2018 Human Development Index, and the new World Bank Human Capital Index ranks Malawi’s harmonized test scores as below average for Sub-Saharan Africa (Malawi obtained a 359 on a scale of 300 – 625). To provide more context to these ratings, Malawi’s 2016 National Early Grade Reading Assessment showed that 91 percent of second graders, and 35 percent of fourth graders in the country could not answer one comprehension question about a short text passage in Chichewa (one of two national languages, along with English) (Nagarajan et. al., 2016). While much of the reason behind these zero scores is poor decoding skills, as demonstrated by oral reading fluency subtask scores that are nearly as low and are highly correlated with comprehension scores, decoding skills are not the only skills that explain the gap.

To identify what other skills’ gaps could be affecting reading comprehension scores and, thus, help the Malawi Ministry of Education, Science, and Technology (MoEST) to determine where students and teachers may need to focus more of their time, MoEST and USAID partnered with Abt Associates to pilot test a new Oral Language Module as part of Malawi’s 2018 National EGRA. The Oral Language Module seeks to identify students’ language competencies, which are critical for achieving reading comprehension competency. To test if zero scores on the comprehension subtask could also be due to issues of speed in reading the short passage (Note: The EGRA comprehension subtask includes five questions, but enumerators only ask questions that correspond with the amount of text that a student was able to read in one minute), we also provided all students who had read at least one word of the oral reading fluency subtask within a minute a second, untimed, opportunity to complete the oral reading fluency task and reading comprehension questions.

In this session, we will present the results of that National EGRA, including findings that showed strong value in the use of the Oral Language Module for differentiating zero scores on the comprehension subtask. We will also share the lessons we learned about the module, including information on which of the four original subtasks worked well and which did not. Specifically, we found value in the inclusion of a subtask with picture prompts, but we found that a subtask that sought to measure vocabulary by asking students to list foods they were familiar with was not at all predictive of reading comprehension scores and had very low internal validity. Finally, we will share information on how we adapted the instrument and adaptation guide based on those findings.

Author

• Melissa Chiappetta, US Agency for International Development (USAID)