Assessment on the AI: Challenges and Opportunities of Leveraging Large Language Model

Session Type: Coordinated Paper Session

Abstract

In an era where Generative Artificial Intelligence (GAI) and Large Language Models (LLMs) like ChatGPT are becoming increasingly integrated into various sectors, this session aims to explore their transformative potential and challenges in the realms of educational and psychological assessment. Bringing together a diverse group of researchers and educators, the session will delve into five groundbreaking papers that offer innovative approaches, methodologies, and solutions for enhancing the reliability, quality, and adaptability of AI-generated outputs in sensitive and critical areas.

The papers cover a wide array of topics, from improving the reliability of language models to automating the process of educational assessment and preventing cheating in online exams. Each paper not only presents novel findings but also discusses the implications for educators, researchers, policymakers, and AI developers.

This session provides essential foundational elements and technologies required for leveraging GAI effectively. Whether you are an educator looking to implement AI in your curriculum, a researcher aiming to push the boundaries of what GAI can achieve, or a policymaker interested in the ethical and practical considerations of AI in education, this session offers the tools and insights you need to navigate the complex landscape of Generative Artificial Intelligence in educational and psychological assessments.

Sub Unit

• Individual Papers, Innovation Demonstrations, Coordinated Sessions, or Organized Discussions

Session Organizer

• Jaehwa Choi, GEORGE WASHINGTON UNIVERSITY

Individual Presentations

• Realtime Intra-Self-Evaluation: Framework for Coping with Hallucinations of Large Language Models - Dayeon Lee, UNIVERSITY OF MARYLAND - COLLEGE PARK

• Assessment Reverse Engineering by AI: A Premiere - Shonai Someshwar, UNC Greensboro; Eunji Lee, UNIVERSITY OF GEORGIA

• Item Modeling for Preventing Assessment Cheating with Technology - Eunji Lee, UNIVERSITY OF GEORGIA

• Enhancing the Distractor Development Process by Leveraging AI Techniques - Sunhyoung Lee, University of Nebraska-Lincoln

• Potential Application of Generative AI in Adaptive Psychological Testing - Yewon Kim, EWHA WOMANS UNIVERSITY; Yejin Woo, Ewha Womans University; Youn-Jeng Choi, EWHA WOMANS UNIVERSITY

Discussant

• Jaehwa Choi, GEORGE WASHINGTON UNIVERSITY