Measuring Information and Communications Technology: A Deeper Look at Computer and Information Literacy and Computational Thinking

• In Event: Moscone Center North Roundtable Session 4
  In Roundtable Session: International Computer and Information Literacy Study 2018: Situating U.S. Results Within an International Context

Abstract

In many countries, there have been increased efforts over time to include information and communication technology (ICT) skills as part of the formal classroom learning environment. Building on the success of the first ICILS (2013) study, ICILS 2018 measured and reported on student computer and information literacy (CIL) achievement across 14 countries. In addition, nine of those countries also measured and reported student achievement in computational thinking (CT), a new, optional module added for the 2018 assessment.

The ICILS 2018 Assessment Framework (Fraillon et al., 2019) articulates the conceptual and methodological frameworks for ICILS and defines the key constructs measured in the assessments (i.e., CIL and CT). CIL is defined as “an individual’s ability to use computers to investigate, create, and communicate in order to participate effectively at home, at school, in the workplace and in society” (Fraillon et al., 2019, p 16) and CT is defined as “an individual’s ability to recognize aspects of real-world problems which are appropriate for computational formulation and to evaluate and develop algorithmic solutions to those problems so that the solutions could be operationalized with a computer” (Fraillon et al., 2019, p 27).

This paper will first discuss the theoretical underpinnings and conceptual framework for the operationalization of CIL and CT as student achievement measures in ICILS 2018. Included in this will be a discussion of the relationship between CIL and CT as distinct but complementary achievement constructs relating to a broader notion of digital literacy.

Following this, the paper will describe how data collected from students in ICILS 2018 were used to create the achievement scales in CIL and CT, using the Rasch Item Response Theory Model (Rasch, 1960), and how these data were combined with descriptions of student achievement to describe hierarchies of achievement in each domain. In CIL these are manifest as four levels of achievement and in CT as three regions of achievement across each scale.

The paper will then describe student achievement in CIL and CT within and across countries. This will include comparisons of distributions of achievement (means, variances, and the proportions of students across levels and regions of each domain). The paper will conclude with a discussion of the relative achievement in CIL and CT across countries and with respect to selected student-level characteristics including gender, socioeconomic status (measured with respect to the number of books in the home, highest level of parental education, and highest level of parental occupation), and their experience using computers. To examine these relationships, correlational analysis will be conducted to investigate to what extent student characteristics are related to CIL and CT outcomes. The paper will also present separate regression analyses of these student characteristics with CIL and CT, respectively, as the dependent variable.

Author

• Julian M.S. Fraillon, International Association for the Evaluation of Educational Achievement