Scenario-Based Assessment (SBAs) in the Age of Gen-AI: Developing Tools for Postsecondary Instructors

Session Type: Demonstration/Performance

Abstract

In this session, we will present first year R&D of a project to create gen-AI enhanced SBA authoring tools for post-secondary instructors. SBAs place knowledge and skills into a scenario or practical context so that test-takers can be both observers of and active participants in their own performance. This creates the opportunity for a reflective, metacognitive, and self-regulatory loop for learners. Instructors can thus use SBAs formatively while maintaining the psychometric properties necessary to evaluate student success and monitor the quality of instruction. Recent advances in gen-AI make it possible to scale-up and democratize SBA development, enabling instructors to design and administer localized, personalized, and discipline appropriate performance assessments that provide better feedback, higher levels of adaptivity, and richer diagnostics.

Sub Unit

• Division J - Postsecondary Education / Division J - Section 2c: Assessing College Student Success and Impact

Chair

• John P. Sabatini, University of Memphis

Participant

• Arthur C. Graesser, University of Memphis

Papers

• Using Scenario-Based Assessment (SBAs) to Deliver Culturally Relevant Test Content - Zuowei Wang, Educational Testing Service; Tenaha P. O'Reilly, Educational Testing Service

• Demonstrating an Authoring Platform for Scenario-Based Assessment - Paul D. Deane, Educational Testing Service; Piotr Mitros, Independent Researcher

• Techniques To Develop Prior Knowledge Assessments To Support Students And Instructors In College Courses - Omer Ari, Georgia State University; Joseph Magliano, Georgia State University; Jonathan Cohen, Georgia State University

• Self-Identified Race, Test Anxiety, Reading Motivation, and Reading Comprehension: A Case for Serial Mediation - Daniel Feller, University of Memphis; John P. Sabatini, University of Memphis