Aligning Measurement with Purpose: USAID’s Revised Approach to Measuring Learning Outcomes

• In Event: Advancing Education Measurement for SDG 4.1.1(a): Improving Assessment Tools, Methods, and Reporting

Proposal

In coordination with the World Bank, USAID contracted RTI in October 2006 to develop the Early Grade Reading Assessment (EGRA) to fill a gap in the measurement of early grade reading outcomes. At the time (and still in several cases), many countries did not have assessments that could measure foundational levels of student learning on a large-scale. While some international and regional assessments do measure these skills, they require complicated scoring/scaling techniques and do not often measure pre-reading skills, resulting in floor effects in many countries. USAID was looking for a short, low-stakes assessment with simplified scoring that countries could easily adapt and use in different contexts to provide teachers and policymakers with information to inform practice. That is how the EGRA was born (RTI, 2015).

Since 2006, EGRA has been adapted into more than 120 languages, used in more than 70 countries (Crouch & Gove, 2019), and even adopted by some countries as their national assessment. Despite its success, EGRA has also often been the subject of critique. While EGRA is an open-source instrument, it must be adapted to the local context and language and leveled to the grade of students participating. Moreover, sampling must be appropriate, enumerators must be well trained and consistent in their administration, and appropriate weights must be applied during sampling. However, not all organizations implementing the EGRA have the same level of expertise and experience. This results in EGRAs of differing quality. Further, there are also critiques of the comprehension module in the assessment, which in its original form, required learners to read as much of a passage as they could in one minute before answering comprehension questions, often conflating speed and fluency with comprehension. Experts also suggested that five comprehension items (the standard number for an EGRA) were insufficient to measure comprehension. Additionally, despite having modules that measure pre-reading skills, many developing countries still found significant floor effects in their EGRA results. Finally, policymakers and donors wanted to use EGRA to compare outcomes across languages and contexts, but this was not possible until 2018, when the Policy Linking for Measuring Global Learning Outcomes (PL) methodology was born.

Since 2006, many new assessments instruments have been created, and a large variety of options now exist for countries and donors to compare learning outcomes across time. USAID still believes EGRA fills a niche that provides value for many countries, but we also recognize the need for improvements. For that reason, we have procured: 1) an EGRA 3.0 Toolkit that will introduce new modules and tasks to help differentiate floor effects and better measure comprehension and 2) a decision tree that will help countries to identify the right assessments and modules for their context and purpose. Because EGRA and PL are both tied to USAID Foreign Assistance Indicators, we are also reconsidering measurement techniques for those indicators. We will present these new tools and details in this panel.

Authors

• Hetal Thukral, DDI Center for Education, United States Agency for International Development (USAID)

• Melissa M Chiappetta, USAID