Search
Program Calendar
Browse By Day
Browse By Time
Browse By Person
Browse By Room
Browse By Unit
Browse By Session Type
Search Tips
Personal Schedule
Sign In
Session Type: Coordinated Paper Session
This coordinated session illustrates five recent developments in automated scoring. Some of the developments are due to the incorporation of transformer models or their variants (Devlin et al., 2019; Vaswani et al., 2017), the refinement of topic identification strategies (Xiong et al., 2021), or technological breakthroughs (Saxton et al., 2019). The first paper shows the challenges and complications of trying to operationalize the scoring of math items. The second uses an mBERT transformer model to evaluate writing in Persian, but can easily be applied to a number of different languages. The third paper highlights how Latent Dirichlet Allocation has improved to help with identifying the topic of writing. The fourth explains the advances in the automated scoring of spoken responses. The last paper is an example of how a traditional operational scoring engine was improved by adopting variations of BERT transformer models. Taken as a whole, these developments highlight that inroads in one area of automated scoring often impact other areas in a positive way. Ultimately, these recent innovations in automated scoring contribute to the broader goal of more equitable and effective educational measurement.
Automated Scoring of Math Constructed-Response Items - Scott Hellman, Pearson; Luis Alejandro Andrade-Lotero, PEARSON; Kyle Habermehl, Pearson; Alicia Bouy, Pearson; Lee Becker, Pearson
Scoring Essays Written in Persian Using a Transformer-Based Model: Implications for Multilingual AES - Tahereh Firoozi, University of Alberta; Mark J Gierl, University of Alberta
Latent Dirichlet Allocation of Essays - Jordan Wheeler, University of Nebraska Lincoln; Shiyu Wang; Allan Cohen, UNIVERSITY OF GEORGIA
Automated Scoring and Feedback for Spoken Language - Klaus Zechner, EDUCATIONAL TESTING SERVICE; Ching-Ni Hsieh, Educational Testing Service
Redesigning Automated Scoring Engines to include Transformer-based Deep Learning Models - Susan Lottridge, Cambium Assessment, Inc.; Christopher Michael Ormerod, Cambium Assessment; Milan Patel, Cambium Assessment, Inc.