A Comparison of Approaches to Automatic Item Generation (AIG) at a Large-Scale Online Competency-Based University

• In Event: Measurement Methods for Complex Problems in Higher Education

Abstract

AIG is a tool for quickly creating large banks of quality items via computer technology, which is useful for addressing scalability and security concerns for online assessments. However, multiple AIG methods exist. The purpose of this paper is to identify the relative advantages of two AIG methods: 1) Natural Language Processor (NLP) and 2) Model-based AIG at a large-scale online university. The results of a series of pilots indicate that the NLP-based approach is most useful for low cognitive level items with simple item stems, whereas the model-based approach is most useful for high cognitive level items with more complex item stems. Moreover, model-based AIG has the unique advantage of using cognitive design to improve quality and target difficulty levels.

Authors

• Heather Hayes, Western Governors University

• Kate Pinder-Baugher, Western Governors University