Automated Item Generation of Reading Inference Test Items: A Natural Language Processing Approach

• In Event: Assessment and Technology

Abstract

This study introduces a novel approach in automated item generation of the previously challenging domain of automated item generation in reading comprehension item generation. The current method disambiguates an underlying subtopic structure from narrative stories—the Harry Potter series—using topic modelling analysis, a weighted Latent Dirichlet Allocation approach. Then, the disambiguated subtopic information is logically combined and arranged using item models from template-based approaches to generate reading inference-type items. This study has the potential to contribute to the methodology and the current practices of automated item generation by highlighting the importance of integrating two primary components–item models and natural language processing techniques–to generate test items in the previously challenging domain of reading comprehension.

Authors

• Jinnie Shin, University of Florida

• Mark J. Gierl, University of Alberta