Assessing the Assessments: Taking Stock of Learning Outcomes Data in India

• In Event: From where you are to where you want to be: How information shapes goals, behaviours and motivation of actors in education

Proposal

Data on learning outcomes is essential for tracking progress in achieving education goals, understanding what education policies work (and don’t work), and holding officials accountable. We assess the accuracy and reliability of India’s two nationally representative surveys on learning outcomes, ASER and NAS. After restricting our sample to maximize comparability, we find that NAS state averages are significantly higher than ASER states averages and averages from an independently conducted nationally representative survey (IHDS). In addition, state rankings based on NAS data display almost no correlation with state rankings based on ASER, IHDS, or net state domestic product per capita. We conclude that NAS state averages are likely artificially high and contain little information about states’ relative performance. We then analyse the internal reliability of ASER data using variance decomposition methods. We find that while ASER data is mostly reliable for comparing state averages, it is less reliable for looking at changes in in state averages, district averages, or changes in district averages.

Author

• Doug Johnson, Independent researcher and consultant