The Logistic Problems of Developing the International Project and Some Partial Solutions

• In Event: The Challenges of Assessing and Comparing Young Children in Different Cultures

Abstract

The paper aims to set out the major logistical problems associated with developing an international study of young children and to set out solutions to those problems where possible. It does this by taking the perspective of a project developer, drawing on theoretical positions related, for example, to child development and test development. But it also draws on the practical experience of working in the field with international collaborators and finding solutions to the challenges that present themselves.
There is scientific interest in children’s development during their first year of school from a variety of perspectives, including cognitive development, social and emotional skills, behaviour and physical development, but a general theme which brings them together is the desire to improve children’s later outcomes. Many studies, mostly within single countries, have shown correlations between cognitive, social, emotional, behavioural traits and motor skills in the early years, and later outcomes (e.g. Bushnell and Boudreau, 1993, Merrell and Bailey, 2012; Tymms et al., 2012; Kautz et al, 2015; Polderman et al., 2010; Washbrook et al., 2013).
Making valid comparisons between young children from different countries and cultures is a challenge. To assess some areas, such as mathematics, we can ask children questions. These can be adapted for different countries and cultures with relative ease, and valid comparisons made, but a series of challenges have been encountered in iPIPS. We identify 10 challenges of which four are set out below:
• Where judgements are made on the basis of observations of children’s behavior, direct equating is inappropriate (see, for example, Merrell et al., 2013). Our solution is to employ video clips of some children’s behaviour which teachers rate before rating their own pupils’ behavior.
• The timing of the assessment: Should this be when children are a certain age, which is the approach taken by some international studies, or at the start and end of the children’s first year of school? We argue that unless data are collected at two time points, at the start and end of the first year at school, valid comparisons are problematic because school has a major impact on cognitive development.
• How should the cognitive assessments be administered? A computer-delivered adaptive assessment has previously been used successfully (Tymms et al., 2012). However, this approach is not possible in some communities. Instead, we have developed an Application for a smart phone which enables an assessor to record data and use an adaptive assessment with minimal training.
• Many expect league tables of countries from international studies but these can be damaging and we have pioneered formative feedback which does not involve rank ordering but nevertheless gives policy makers and participants valuable feedback.
The results of successfully trying to create solutions to the problems of developing an international study of children starting school, lead to the conclusion that the issues which have been thrown up are largely surmountable. An international project is hard but possible
This conclusion is of considerable importance since it opens the door to a major project of significant scientific worth.

Authors

• Christine Merrell, Durham University

• Peter B Tymms, Durham University