A Comparison of Automatically Generated and Human-Written Items at an Online Competency-Based Higher Education Institution

• In Event: Marriott Roundtable Session Eight
  In Roundtable Session: Assessment in an Online Context (Table 6)

Abstract

For online competency-based higher education institutions that offer degrees and microcredentials across a wide range of disciplines, it is necessary to develop and maintain hundreds of summative assessments with the utmost quality and security. Automatic item generation (AIG) is a powerful tool for quickly developing a large item bank for replacing overexposed, leaked, and underperforming items on an assessment. The purpose of this study was to compare the quality of items developed via AIG to that of human item writers for a diverse group of high-volume undergraduate course assessments within Western Governors University. AIG items generally outperformed human-written items in course congruency and statistical quality while minimizing development time and costs, justifying its use (albeit with caveats).

Authors

• Heather Hayes, Western Governors University
  Presenting Author

• Kate Pinder-Baugher, Western Governors University
  Non-Presenting Author

• Josie Rodriguez, Western Governors University
  Non-Presenting Author