AI Literacy Proxy from PISA 2022 Hong Kong: IRT and SEM Validation with ICT Capabilities

• In Event: AERA Poster Session Wednesday 11:45am
  In Poster Session: Poster Session I -- AERA Division D Section 1: Measurement, Psychometrics, and Assessment

Abstract

This study proposes and validates a proxy index of Artificial Intelligence (AI) literacy using student questionnaires from PISA 2022 Hong Kong. AI Literacy Proxy aligned with four theorized dimensions (Awareness, Usage, Evaluation, and Ethics), which were screened across instruments and calibrated with a unidimensional Rasch partial‑credit model. A three‑dimensional Rasch model estimated ICT Access, Use, and Competence. Structural equation modeling (SEM) revealed strong direct effects of AI literacy on ICT Use and Competence, and a moderate effect on Access, with limited mediation through Access or Use once AI literacy was included. Findings suggest conceptual AI understanding is a foundational determinant of students’ digital capabilities and provide a replicable approach to proxying AI literacy in large‑scale assessments lacking direct measures.

Authors

• Linwei Yu, The University of Hong Kong

• Yueru Li, University of Macau