Exploring the Role of CD-CAT in Gateway STEM Learning with Generative AI

• In Event: Toward a National Infrastructure for Equitable, AI-Enhanced Learning Diagnostics in STEM Education

Abstract

Abstract
Introductory STEM courses at large U.S. universities are often delivered in massive lecture formats, enrolling over a thousand students annually. This “one-size-fits-all” model, typically constrained by limited instructional resources (e.g., teaching assistants and graders), contributes to high DFW (D, F, or Withdrawal) rates—often between 30% and 50%. These rates are even higher among underrepresented minority (URM) students; for example, a recent Algebra and Trigonometry course at a Midwestern university reported a 66% DFW rate for URMs. As foundational “gateway” courses, failure in these subjects can derail students’ progress in STEM majors, leading to major changes or extended time to degree completion.
To address these challenges, researchers developed diagnostic tools based on Computerized Adaptive Testing (CAT) and Cognitive Diagnostic Models (CDMs). These tools deliver personalized assessments and targeted feedback, enabling instructors to identify specific learning gaps while also detecting and mitigating items with Differential Item Functioning (DIF) to promote fairness.
Over the past five decades, CAT has evolved significantly, transitioning from high stakes testing to broader educational applications. Its adaptive nature makes it well-suited for personalized learning environments. However, building a robust CAT system requires advanced psychometric knowledge and technical expertise—resources often beyond the reach of typical STEM educators.
This presentation demonstrates how to design a reliable and cost-effective CD-CAT diagnostic tool that classifies students’ mastery levels across key cognitive skills. We also explore how integrating Generative AI—such as AI agents—can enhance the system by providing clear, personalized follow-up guidance and a user-friendly interface for educators. More specifically, with the support of the proposed AI-Agent application, instructors can independently assemble item banks with minimal assistance from psychometricians and validate individualized diagnostic testing tools. These tools offer precise insights into student achievement levels and deliver formative feedback to students, educators, departments, and colleges, enhancing instructional decision-making and learning outcomes. Together, these innovations offer a scalable solution to improve equity and learning outcomes in gateway STEM education.

Author

• Hua-Hua Chang, Purdue University