System (in)coherence: Quantifying the alignment of primary education curriculum standards, examinations, and instruction in two East African countries

• In Event: Learning from research on implementation at scale: Implementing large scale instructional improvement programs in low and middle-income countries

Proposal

Improvements in instructional coherence have been shown to have large impacts on student learning at scale, yet analysis of such coherence, especially in developing countries and at a systems level, is rare. Diagnosing areas of incoherence can identify key constraints to improving learning at scale across an education system. For example, if curriculum is incoherent with children’s learning levels, by being overambitious or progressing too quickly (Beatty and Pritchett, 2015; Kaffenberger and Pritchett, 2020; Muralidharan and Singh, forthcoming), then bringing it into coherence with children’s learning, at scale, could potentially achieve large learning gains. Alternatively, if national exams and curriculum are poorly aligned with each other, and teachers are primarily accountable for exam pass-rates, then improvements to curriculum, even if done at scale, may produce little additional learning.

We use an established methodology, the Surveys of Enacted Curriculum (Blank, Porter & Smithson, 2001; Porter, 2002; Polikoff, Porter & Smithson, 2011), and apply it to a developing country context to systematically analyze and quantify the content and coherence of the primary curriculum standards, national examinations, and actual teaching delivered in the classroom in Uganda and Tanzania. We find high levels of incoherence across all three instructional components. In Uganda, for example, only four of the fourteen topics in the English curriculum standards appear on the primary leaving exam, and two of the highest-priority topics in the standards are completely omitted from the exams. In Tanzania, only three of fourteen English topics are covered on the exam, and all are assessed at the “memorization” level. Rather than aligning with either the curriculum standards or exams, teachers’ classroom instruction is poorly aligned with both. Teachers tend to cover broad swathes of content and levels of cognitive demand, unrelated to the structure of either the curriculum standards or exams. An exception is Uganda mathematics, for which standards, exams, and teacher instruction are all well aligned.

The study has served as a diagnostic tool for the education ministries in both countries to identify instructional incoherences and inform potential reforms. The results also point to potential constraints to programs hoping to improve learning at scale. Operating in systems where incoherence is widespread, which is likely the case in countries far beyond our two study countries, means the impact of improvements to one component may be constrained by incoherence of another component. This study illustrates how improving learning at scale may require dynamically improving multiple instructional components to bring them into alignment with each other. In addition to providing empirical results for Uganda and Tanzania, this study provides a proof-of-concept for the use of the SEC methodology as a diagnostic tool, helping education systems identify areas of instructional (in)coherence and informing efforts to improve coherence for learning.

Authors

• Michelle Kaffenberger, RISE Programme

• Julius Favourite ATUHURRA, International Christian University