Innovations in the Measurement of 21st Century Skills

Session Type: Coordinated Paper Session

Abstract

Developing and measuring skills beyond the traditional core domains in education, such as 21st Century Skills (21CS), can enhance diversity and equity in education and the workforce by offering a wider range of strengths for individuals to capitalize on in pursuit of their best possible future. It is crucial for students to develop and master 21CS, and assessment plays a vital role in providing insights into their proficiency and progress. Our session delves into cutting-edge advancements in the evaluation of 21CS, both for individual learners and groups. We begin by broadening our perspective on 21CS in the context of generative AI, contemplating current frameworks and the possible integration of additional skills, as well as measurement considerations. Next, we examine a new multidimensional approach for measuring and nurturing creative thinking skills in a play-based learning environment. We then investigate innovative findings on the stability and generalizability of individual collaborative problem-solving skills across various task types. Additionally, we explore how Cognitive Diagnostic Modeling can be employed to gain insights into student mastery of 21CS within existing assessments. Lastly, we will discuss the potential of AI in streamlining the labor-intensive and time-consuming aspects of assessment development, particularly in generating critical thinking test items.

Sub Unit

• Individual Papers, Innovation Demonstrations, Coordinated Sessions, or Organized Discussions

Session Organizer

• Kristin Lansing-Stoeffler, ACT, INC.

Chair

• Alina A. von Davier, Duolingo

Presenters

• John Weiner, Lifelong Learner Holdings

• Jiangang Hao, EDUCATIONAL TESTING SERVICE

• Yigal Rosen, BrainPOP

• Kristin Lansing-Stoeffler, ACT, INC.

Individual Presentations

• Reexamining 21st Century Skills in the Era of Generative AI - John Weiner

• A Multidimensional Method for Measuring and Fostering Creative Thinking Skills in Learning-through-play Setting - Yigal Rosen, BrainPOP; Garett Jaeger, LEGO Foundation; Michelle Newstadt, BrainPOP; Sara Bakken, BrainPop; Ilia Rushkin, BrainPOP

• Characterizing Individual Collaborative Problem-Solving through Conditional Transition Profile: Stability across Tasks and Teams - Jiangang Hao, EDUCATIONAL TESTING SERVICE

• Hiding in Plain Sight: 21st Century Skills in Existing Assessments - Kristin Lansing-Stoeffler, ACT, INC.; Andries Van der Ark, RESEARCH INSTITUTE OF CHILD DE; Tessa van Schijndel, University of Amsterdam; Jeffrey Steedle, Curriculum Associates; Alina A. von Davier, Duolingo

• Harnessing Artificial Intelligence for Generating Items in Critical Thinking Tests - Alina A. von Davier, Duolingo; Hyo Jeong Shin, Sogang University; Helen Baron, SAVILLE & HOLDSWORTH LTD; Ji Hoon Ryoo, Yonsei University

Discussant

• Ahmad Rahimi, University of Florida