Adolescent Test-Taker Characteristics: A Qualitative Validation Study of a Chinese Language Reading Examination in Singapore

• In Event: Centering Multilingualism for More Equitable Education Systems in Assessment: Global Perspectives

Abstract

Objectives
This qualitative validation study investigates the characteristics and needs of adolescent test-takers in multilingual Singapore sitting the reading paper of the Singapore-Cambridge General Certificate of Education Ordinary-Level Chinese Language Examination (GCE 1160). Specifically, evidence gathered through semi-structured interview and document analysis was organized around the following three issues of multilingual adolescents’ literacy, reading motivation, interests and authenticity, and new forms of reading literacy.

Theoretical Framework
Among the many validation frameworks, Weir’s socio-cognitive framework (Khalifa & Weir, 2009) has been extensively used across a range of validation studies in language testing and assessment, including studies conducted by Cambridge Assessment English for its suite of English as a second language examinations. The framework comprises six validation components, namely test-taker characteristics, cognitive parameters, contextual parameters, scoring, criterion-related, and impacts on teaching and learning. As addressing all six components in the framework is beyond the scope of a single study, this study focuses solely on test-taker characteristics, whilst noting its relation with other components in the framework.

Data sources/methods
The study draws on semi-structured interviews and document analysis. Across the pilot and main studies, a total of 22 multilingual stakeholder interviewees were selected using a purposive maximum variation sampling technique (Patton, 2015). The interviewees included an elite policy group with privileged access to the detailed test specifications and procedures. Interviews were also carried out with Chinese as a second language teachers and secondary school students, whose perspectives are seldom considered in validation processes. In addition, opinions were sought from experts in the field of reading and assessment.

Subsequently, findings from the semi-structured interviews were compared with those gleaned from the analysis of documents. Documents examined are held or published mainly by the Ministry of Education, the Singapore Examinations and Assessment Board and Cambridge Assessment English. The selection of documents included recent Chinese language and mother tongue languages reviews, reports, press releases, speeches, and presentations.

Findings/scholarly significance
It was observed that Singaporean multilingual adolescents are generally becoming less motivated to read extensively in the Chinese language, seemingly the direct result of digital technologies and heavy homework loads. It also appeared that motivation to read is correlated with the relevance and appeal of texts available as well as the perceived value of reading in securing better education and job opportunities. Another pattern illustrated by the study was that the Internet is the defining technology for literacy and learning for Singaporean adolescents. Whilst evidence suggested that the reading examination is designed with adequate knowledge of adolescence and adolescent literacy, several threats to validity were identified, including the relatively low appeal of the passages to test-takers, their relevance, authenticity, and sensitivity to new forms of reading literacy. Drawing on Weir’s socio-cognitive validation framework, the study analyses these threats to validity and outlines practical directions whereby policymakers and test designers might develop the reading examination.

Author

• Yun Yee Cheong, Singapore Ministry of Education