Title: Estimating typical annual learning gains in early grade reading: Evidence from 13 countries

• In Event: Assessing student outcomes to inform effectiveness, estimate typical learning gains and the impact of high-stakes testing

Proposal

We, as a community, are sitting on a treasure trove of early grade reading data. However, we do not always use these data to their full potential. Now, perhaps more than ever, we should be using these data more collectively to better understand trends across countries and learning patterns around the world, to help us better understand the impact of COVID-19 on learning outcomes for children. With that in mind, we recently examined datasets from thirteen countries (see list below) that contained reading assessment data on students in grades one, two, and three. Specifically, we looked at data from cross-sectional studies that assessed students in consecutive grades (grades 1 and grades 2 or grade 2 and grade 3) using the same passage, in the same schools, at the same point in the academic year. Since these evaluation data were collected at the end of the year, they allow us to estimate a typical annual grade gains in student performance. For example, a study that assessed grade 2 and grade 3 students at the end of the academic year is able to show what a grade 3 learning gain looks like in terms of reading fluency (i.e. the growth from end of grade 2 to end of grade 3).

When plotting the distributions for the consecutive grades in each of these datasets, we were struck by the parallel tendencies. In nearly every case, we saw that expected gains were strikingly similar for students at nearly every part of the distribution. We also found that a typical grade 2 gain was between 11-15 words per minute (average 14.4) and a typical grade 3 gain was also between 11-15 words per minute (average 13.5). This presentation will focus on the use of grade distributions across various countries (e.g. cumulative distribution, mean, standard deviation, and inter-class correlation), in order to provide answers to the following questions:

- Using grade 3 reading fluency distributions alone, how well can we predict grade 2 distributions and thus predict expected “grade gains” and COVID-related “grade losses” in a given population?
- What factors are most essential for developing reliable grade gain models (e.g. mother tongue vs second/third language, lower-order reading skills, etc.)?

By answering these questions, we hope to help ministries of education, donors, implementing partners, and other stakeholders better understand what could be expected from their students in a given year. Particularly when evaluation data are only collected information from one grade. These results can also be used to begin to understand expected losses stemming from COVID-related school closures. We also hope that this work will inspire others to replicate and refine these models using data from additional countries, leading to a more comprehensive and robust study.

Countries currently included: Cambodia, Egypt, El-Salvador, Ethiopia, Iraq, Jordan, Kenya, Liberia, Nepal, Philippines, Tanzania, Uganda, and West Bank

References:
Cummiskey C, Stern J Calculating the Educational Impact of COVID-19 (Part II): Using Data from Successive Grades to Estimate Learning Loss. May 13,2020. https://shared.rti.org/content/calculating-educational-impact-covid-19-part-ii-using-data-successive-grades-estimate

Authors

• Chris Cummiskey, RTI International

• Jonathan Stern, RTI International