Assessing the suitability of Group Administered Formative Assessments for large primary classrooms in Tanzania: lessons from an implementer’s pilot study

• In Event: Reduce education injustice and social exclusion through curriculum and practice

Proposal

Education research from a variety of contexts well documents that teachers oftentimes overestimate their students’ skills and learning achievement on assessments (Djaker, S., Ganimian, A. J., and Sabarwal, S., 2022). Regular classroom-based formative assessments help provide teachers with invaluable data to accurately inform them about their student’s learning and skills thereby enabling them to better adapt instruction to ensure more equitable learning outcomes (Kim et al., 2016; Tsagari, 2016). For assessing early grade students’ literacy skills, teachers, who are supported by the Implementer, are encouraged to use individually administered formative assessment tools that assess each students’ letter sound knowledge, decoding, oral reading fluency and comprehension skills among others (Implementer, 2022). However, the administration and use of individually administered formative assessments in Tanzania among other countries in the Global South, are oftentimes hampered by large classroom sizes, demand to cover ambitious curriculum and a lack of time to carefully analyze and respond to the results. In response to these challenges, another implementer, RTI International has explored the use of group-administered learning assessments as a potential solution in several countries including Ghana, Egypt, Tanzania and Ethiopia with initial promising results relating to the tool’s reliability (Stern, 2017).

Considering these challenges and the prior research, the Implementer has undertaken a two-part research study to assess the suitability of using the Group Administered Learning Assessments (GALA) as an alternative approach to conducting individual formative assessments with primary grade students in its supported schools in Tanzania. The first part of the study, a small-scale pre-pilot was conducted with Grade 1 and 2 students in three schools in Tanga Province in 2021. Preliminary findings from the pre-pilot suggest that the group-administered learning assessments can be implemented with proper coordination and training, but it can be time-consuming and laborious, especially at the initial stages of implementation. In addition, the administration procedure did appear to have an influence on student performance, especially regarding the non-response rate (Implementer, 2022).

Building upon these initial findings, the Implementer further adapted the administration procedure before conducting the full-scale pilot study with a larger sample of Grade 1 and 2 students in five schools in Tanga province in July 2022. During this pilot study, data was collected from a total of 768 students on several foundational literacy skills tasks that aligned to the Tanzanian curriculum, including letter sound recognition, non-word recognition, oral reading fluency, reading comprehension and a paired sentence choice subtask. Three types of assessments were administered: Process 1, a group administered learning assessment with all children in each class; Process 2, an individual oral administration to 50% of the children who participated in the Process 1 (n=361), and Process 3, an individual written administration of the GALA to the other 50% of children who participated in process 1 (n=360). To better understand the administration experience of the GALA for the various participants and stakeholders, the Implementer also collected data through a classroom observation tool, a student experience survey with a total of 100 children across the schools and grade levels, and a teacher experience survey to better understand how the procedure may fit within their existing instructional and formative assessment practices. Lastly, video recordings of the GALA administration were also conducted to provide additional insights to support decision-making.

This paper presents the findings from this larger study, as well as on-going discussion questions regarding the GALA’s utility and feasibility. Specifically, the full-scale GALA pilot seeks to understand: 1) What are the experiences and perceptions of administrators, students, and teachers of administering the GALA as an alternative to individual oral formative assessments? 2) How do students’ response rates and literacy skills compare across the three different administration procedures (group written, individual oral and individual written)? 3) What are the trade-offs (pros and cons) in terms of types of individual learning achievement data that these different assessment procedures offer to teachers, and other stakeholders? Using the results from these research questions, the Implementer ultimately aims to assess whether the GALA offers teachers and education systems a suitable alternative approach to conducting individual formative assessments to inform instruction in the context of large class sizes in Tanzania and in other similar contexts in which it works.

While the data analysis is currently in-progress, preliminary findings from the full-scale pilot suggest that the GALA may offer education systems a more efficient alternative to conducting individual assessments. Furthermore, given it was largely received positively by the sampled Grade 1 and 2 teachers, it seems feasible that with adequate training and support, the GALA could be integrated into existing instructional and assessment practices within the early grade classrooms that the Implementer supports. However, there are several limitations to the types of skills that the GALA can assess reliably to inform instruction. As such, implementers and education systems may consider how they can use GALA in conjunction with sample-based individually administered formative assessments to balance the need for efficiency while also ensuring teachers have a robust understanding of their students’ foundational literacy skills. Through this presentation, the Implementer will present several topics such as those listed below for further discussion from the CIES community:

1. Comparability of assessing foundational literacy skills using receptive (listening) vs. productive methods (orally) for letter sound recognition and non-word reading.
2. How important is it to specifically assess oral reading fluency in addition to reading comprehension and other foundational skills on a regular basis?
3. How, if at all, could the Implementer, utilize both the GALA in conjunction with traditional individually administered formative assessments to inform instruction across the schools and classrooms in which it supports?

Authors

• Rebecca Alice Westbrook, Room to Read

• Pinaki Joddar, Room to Read