Equating International Performance Indicators in Primary Schools (iPIPS) Measures Across Different Countries and Cultures

• In Event: The Challenges of Assessing and Comparing Young Children in Different Cultures

Abstract

The aim of the presentation is to assess the possibility of comparing iPIPS data measuring the children’s cognitive development as well as children’s progress during the first year of schooling in England, Scotland and Russia. Is it possible to equate iPIPS measures on a large scale across countries with different curricula and differing traditions? The question is tackled using Rasch measurement approaches.
iPIPS (international Performance Indicators in Primary Schools) was proposed a few years ago as a new international comparative study of children starting school. But the published comparisons of children starting school using this instrument only involved English-speaking countries on a large scale (Merrell and Tymms, 2007). Now the Russian version of iPIPS has been developed and, following previous analyses of small scale trial data, this paper focuses on the results of mathematics for children from the three countries.
The Scottish and English representative samples consist of more than 6,000 children each. The Russian sample consists of more than 3,000 children recruited from two regions of the Russian Federation. The randomised stratified samples are representative of these regions. Drawing on data on about 15,000 children across three countries and across two cycles of assessment (start of year and end of year), we examine whether the tests are cross-culturally valid and free from bias; in other words – can they be equated?
The involvement of a sample from Russia with its different language, culture and traditions and where children are, on average, two years older than children in England and Scotland at the start of school, presented a challenge. Thus, the main purpose of this study is exploratory and it aims to extend our understanding of the possibilities of making comparisons across countries of young children’s development.
Firstly, we examine the test equivalence of different language versions of iPIPS assessment. We used two approaches: (a) analysis of test items and tests under Item Response Theory (IRT) (comparisons of item characteristics, item maps, item hierarchy, dimensionality for two language versions); and (b) identification of Differential Item Functioning (DIF) across countries and within country variables.
Secondly, the possibility of the common scale construction is investigated. Simultaneous Rasch calibration (Wolfe, 2004) is used, that allows us to perform both equating between countries and equating between the two cycles of assessment.
The results show that it is indeed possible to equate iPIPS math measures from the start and the end of the first year at school across Scotland, England and Russia. It allows us to compare what children in the three countries know and can do when they start school and make valid comparisons of children’s achievement from different countries at the end of the first year at school.
This paper is of scientific value as it shows that comparisons of children’s mathematics development is possible across very different cultures and opens up the possibility of similar work in other countries.

Authors

• Elena Kardanova, National Research University High School of Economics

• Alina Ivanova, Higher School of Economics, Russia

• Peter B Tymms, Durham University