Individual Submission Summary
Share...

Direct link:

Innovations in test development: Automatic Item Generation for LSA

Tue, April 16, 5:00 to 6:30pm, Hyatt Regency, Floor: Atrium (Level 2), Garden Room A

Proposal

Collecting data about students’ knowledge, core competencies and cognitive processes that build the basis for the acquisition of additional and new knowledge has been generally done by means of traditional methodology where an assessment instrument is developed using teams of experts in the different stages of the process, who create item specifications that allow for the development of a small number of items at a time. This is a complex, expensive and sometimes inefficient process that takes time, many resources -economic, chronological and human- and yields a limited amount of test items that eventually have to be discarded because of the wearing effects. In comparative education studies, some of the biggest challenges in the field of assessment are related to the selection of the type of information that is common to different countries’ educational systems, curriculum and core knowledge. Assessments need to be improved in order to make this process more efficient. Automatic Item Generation (AIG) is an innovative alternative approach that helps to expedite the development of large-scale tests because it accelerates the production of high-quality test items. Assessment instruments that are developed by means of AIG allow for the creation of pools of high-quality items large enough to generate parallel forms of tests on the fly by using computer software.
The purpose of this paper is to present information on the basic concepts and principles of AIG and the development process of assessment instruments by describing different forms of item models that allow for the generation of large amounts of constructed and semi-constructed response sibling items, and to illustrate how AIG can be used for the development of large-scale student assessments, by presenting a large-scale instrument that was developed by a team of researchers in Mexico to assess students’ core knowledge and general abilities from K-12. By reviewing the literature in the field of AIG and the examples included in this study, a discussion of the advantages of this field over traditional methods and the psychometric challenges it poses over the validity of assessment results will be presented.

Author