Implementing Exams Based on Item Response Theory in Higher Education

• In Event: Exams, Grades, and Assessment Methods in Higher Education

Abstract

University exams deal with known weaknesses in psychometric qualities jeopardizing the validity of score interpretations. In this contribution, we introduce a study on the development of an IRT-based exam in a teacher education lecture. With a sample of N = 673 students and an item pool of 40 MC items, we explored the model fit (dichotomous vs. polytomous 1PL model), selected high quality items, and investigated possible mode effects (paper-pencil vs. computer-based). We found the dichotomous Rasch model to show the best model fit, and selected 29 items, based on difficulty, item fit, and discrimination. An effect in favor of an assessment mode was not found in general and implications for further item development could be obtained from the analyses.

Authors

• S. Franziska C. Wenzel, Goethe University

• Sabine Fabriz, Goethe University

• Holger Horz, Goethe-Universität Frankfurt