Highlighted Presidential Session: An International Perspective on Digital Literacy - The International Computer and Information Literacy Study 2023

Group Submission Type: Highlighted Presidential Session

Description of Session

Overview
The International Association for the Evaluation of Educational Achievement (IEA) International Computer and Information Literacy Study (ICILS) is the only international large-scale assessment that focuses solely on digital literacy education and provides measures that can be used to monitor changes in students’ digital literacy achievement over time. In addition to measuring student achievement in computer and information literacy (CIL), and computational thinking (CT), ICILS collects extensive targeted data relating to the contextual factors associated with students’ digital literacy learning in school and outside of school.
This session will provide an overview of ICILS 2023 and will report key findings related to students' achievement in CIL and CT. It will further report on the contexts in which students' CIL and CT learning takes place within and across countries, and on the relationships between aspects of students’ background and ICT use, and their CIL and CT achievement. The session will include reflections on the consequences of these outcomes with respect to the teaching and learning of digital literacy-related competencies, and will provide detail of future planned research using ICILS data and discuss themes for research in the next cycle of ICILS (ICILS 2028).
Introduction
IEA has been studying the relationship between ICT and educational processes, as well as factors related to the pedagogical use of ICT, since the late‐1980s (Pelgrum & Plomp, 2011). IEA’s ICILS emerged in response to the increasing value being placed on the use of ICT in modern society and the need for citizens to develop relevant capabilities to participate effectively in a digital world.
The first cycle of ICILS in 2013 (ICILS 2013) assessed students’ computer and information literacy (CIL) with an emphasis on students’ ability to use computers to collect and manage information and to produce and exchange information (Fraillon et. al, 2014).
The second cycle of ICILS (ICILS 2018) included, as an international option, an additional assessment of computational thinking (CT) with an emphasis on students' ability to formulate solutions to real-world problems so that those solutions could be operationalized with a computer (Fraillon et. al, 2020).
ICILS 2023 includes both the core assessment of CIL and the optional assessment of CT. ICILS 2023 provides, across relevant countries, the opportunity to report on trends in students’ CIL achievement across 10 years and the three assessment cycles since ICILS 2013, and between the 2018 and 2023 assessment cycles in CT.
In addition to measuring variations in CIL and CT among and within countries, ICILS 2023 reports on the relationships between CIL and CT, as well as the relationships between those constructs and students' background characteristics, their access to, and attitudes towards ICT, and their use of ICT both within school, and outside of school.
ICILS 2023 was based around research questions that focused on the following for CIL (in all countries) and CT (in countries where CT was also assessed):
• Variations in students' CIL and CT within and across countries in 2023, and in comparison to previous cycles of ICILS;
• Aspects of classrooms, schools, and education systems that are related to students' CIL and CT
• Aspects of students’ experiences of using ICT, within school and outside of school, that are related to their CIL and CT;
• Aspects of students’ personal and social backgrounds (such as gender and socioeconomic background) that are related to their CIL and CT; and
• The relationship between CIL and CT.
Data collection in ICILS 2023
Participants
The samples were designed as two-stage cluster samples. Schools with enrolled students at the target grade were randomly selected with a probability proportional to size as measured by the number of students enrolled in a school. Students and teachers were then sampled from within the schools sampled at the first stage.
ICILS 2023 collected data from 132,889 grade 8 (or equivalent) students across 34 countries and one benchmarking participant. These students were from 5,299 schools in 34 countries and one benchmarking participant. These student data were augmented by data from 60,835 teachers in those schools, and by contextual data collected from school ICT coordinators, principals, and national research centers.
Data were collected in Northern Hemisphere countries in the first-half of 2023, and in the second-half of 2023 in Southern Hemisphere countries.
Instruments
All ICILS instruments were delivered on computer. The student assessment instruments, bespoke developed for ICILS, were designed to reflect contemporary software application conventions, such as the use of recognizable icons associated with typical functions, or common user interface feedback responses to given commands.
The ICILS 2023 CIL assessment instrument comprised seven 30-minute assessment modules. Each student completed two of the seven modules. The modules comprised a set of questions and tasks based on a real-world theme and follow a linear narrative structure.
The CT assessment instrument comprised four 25-minute assessment modules. Each student completed two of the four modules. Each CT module comprised a set of questions and tasks relating to real-world problems that may be addressed with computer-based solutions.
Data relating to the background of participants in ICILS and contexts in which CIL and CT are being learned were collected using:
• A 30-minute questionnaire completed by all students;
• A 30-minute questionnaire completed by a random sample of 15 grade 8 teachers in each sampled school;
• A 20-minute questionnaire completed by the designated ICT coordinator in each sampled school;
• A 20-minute questionnaire completed by the principal of each sampled school; and
• A national contexts survey completed under the oversight of the national research center in each participating country.
As an international option, principals in 12 countries provided information about their responses to the use of generative AI (such as ChatGPT) in their schools, and their perceptions of the likely impact of the use of generative AI on the work of students and teachers.
Results
Key results of ICILS 2023 will be reported in this session. The session will report student achievement, as measured and described by the CIL and CT proficiency scales. The session will further report on the contexts in which student CIL and CT learning are taking place, within and outside of schools, and reflect on these outcomes with respect to the broader contexts for the teaching and learning of CIL and CT. Following each cycle of ICILS, IEA releases an international database of ICILS data to support scholarly research and policy development relating to digital literacy education within and across countries. In the session we will reflect on key findings of ICILS 2023, and discuss some possible areas of future research using ICILS data.
References
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: The IEA International Computer and Information Literacy Study international report. Springer Nature.
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). Preparing for life in a digital world: IEA international computer and information literacy study 2018 international report. Springer Nature.
Pelgrum, W. J., & Plomp, T. (2011). IEA assessments of information and communications technologies (ICT). In C. Papanastasiou, T. Plomp, & E. Papanastasiou (Eds.), IEA 1958‐2008: 50 years of experiences and memories. Culture Center of the Kykkos Monastery.

Sub Unit

• General Pool

Chair

• Oren Pizmony-Levy, Teachers College, Columbia University

Plenary Speaker

• Julian Fraillon, Stichting IEA Secretariaat Nederland

Discussant

• Sobhi Tawil, UNESCO