Is a Universal Learning Scale Feasible? Lessons from existing comparative school surveys

• In Event: Universal Learning Scale: Conceptual feasible, operational doable?

Proposal

Objectives: The work of UIS, through the Global Alliance to Monitor Learning, to define a learning scale for the monitoring of SDG target 4.1 and contribute to other learning related targets is an essential step in bringing life to the ambitions for education progress in the 2030 Agenda for sustainable development. Despite the need for such robust comparative data on learning, it is hard to overstate the technical challenges of defining and developing a global comparative learning scale. Moreover, added to the standard technical challenges of producing and collecting a valid and comprehensive measure of learning, is the challenge to adhere to the ‘no-one left behind’ principle of the 2030 agenda. The objective of this paper is to open a discussion child coverage and representation in new data collection or harmonization of existing surveys drawing from evidence on the who’s left behind in existing surveys of school children, and what this means for the monitoring and measurement of education progress for all children.

Perspective: The authors’ perspective comes from experience of evaluating the potential for international school based surveys of children – including learning assessments (PISA, TIMSS, PIRLS) and health behaviour surveys (HBSC and ESPAD) – to inform child policy and monitor child well-being in Europe for a joint European Commission and OECD project (Richardson and Ali, 2014).

Methods: The evidence presented in the paper is drawn from an in-depth review of, and evaluation of possible systematic bias in, selected international school based surveys to assess the representability and reliability of learning metrics (and children’s responses to other items) included in these studies. The review assesses the extent of coverage of all children within these sources and, using various multivariate techniques, the evaluation of bias assesses how differences in systems of education, linguistic differences, and differences in non-response by items included might skew the results. The paper concludes with a reflection of the data for cross-national monitoring of children’s learning and well-being can be improved, and methods for facilitating harmonization and greater representability in existing collections.

Results: The results of the study have shown that existing school based surveys of children require adaptation in order to be fully fit for learning metrics that aspire to inform a global agenda for improving learning where ‘no-one is left behind’. These surveys do not presently capture many young children, children who are out of school, or children who are in school but have learning needs. Moreover, evidence from the tests of bias in non-response of survey participants highlight concerns that has the potential to affect how a learning scale might simultaneously be used to inform policy and practice. Updated tests on more recent cross-national school surveys will also be presented (including, if possible, from other regions of the globe).

Significance: Policy responses in recent years to international education studies such as PISA show that any successful effort to populate SDG education targets with a learning scale will contribute to changes in policy and practice in attempts to improve outcomes on this learning scale. To ensure that these efforts are not detrimental to the ambition of leaving no-one behind, it is important that these data sources are sensitive to the situation of all children, either through comprehensive data collection, or clear acknowledgement of whose learning experiences are absent from the scale. Moreover, data on children’s learning outcomes will feed in to public debates on education policy, and in turn about how to spend national education budgets most effectively and efficiently. If learning statistics are not-representative / comprehensive, limited educational expenditure is at a risk of being wasted to the detriment of children and society as a whole.

References
Richardson, D. and N. Ali (2014), “An Evaluation of International Surveys of Children”, OECD Social, Employment and Migration Working Papers, No. 146, OECD Publishing. http://dx.doi.org/10.1787/5jxzmjrqvntf-en

Author

• Dominic Richardson, UNICEF