The Effect of Mode and Keyboard Input Method on Reading Performance: Evidence From 2021 PIRLS Data

• In Event: Research on Cognitive, Reading, and Resilience Outcomes in Large-Scale Assessments

Abstract

International large-scale assessments, such as Progress in International Reading Literacy Study (PIRLS), is shifting from paper-based test to digital test. This present study aims to investigate a potential mode effect (paper-based vs digital) in such assessment, and whether different keyboard input methods influenced reading comprehension performance. We analyzed 26 countries’ data in 2021PIRLS using Item Response Theory. Results showed a significant mode effect - paper-based results overall outperformed digital ones. More importantly, there was also a significant keyboard input effect – countries whose assessment language has a lower consistency (e.g., Chinese) between orthographic system and its representation on the keyboard scored lower than those with a higher consistency (e.g., English). Findings lead to the discussion on large-scaled assessment and educational implications.

Authors

• Jiahuan Zhang, The Hong Kong Polytechnic University

• Run Mu, University of Hong Kong

• Wai Ip Lam, University of Hong Kong

• Chung Pui Tai, University of Hong Kong

• Ken Chow, University of Hong Kong

• Choo Mui Cheong, University of Hong Kong