Fewer Questions, Sharper Insights: Measuring College Preparedness Using Item Response Theory

• In Event: AERA Poster Session Wednesday 11:45am
  In Poster Session: Measuring What Matters: Validating Constructs of Readiness and Global Competence

Abstract

College access programs aim to improve postsecondary outcomes but often lack formative evaluation to aid implementation. This study applies Item Response Theory (IRT) to evaluate a knowledge-based quiz administered to freshmen in California's EAOP program, which serves predominantly Latinx and low-income students. Analysis of results including model fit, question difficulty and item information support the quiz’s validity as a measurement tool. Results showed that several items had poor discrimination or lacked difficulty, allowing administrators to improve future quizzes. The study highlights the potential of IRT to provide formative assessments in college readiness programs, offering psychometric insights that traditional binary outcome models lack. Recommendations are made to enhance future quiz design and implementation in similar program contexts.

Authors

• Holden Tibbetts Ellis, University of California - Santa Barbara

• Travis Rodolfo Candieas, University of California - Santa Barbara