Applications of Artificial Intelligence and Natural Language Processing in Educational Measurement

• In Event: Harnessing the Power of AI in Large-Scale Assessment: Advancing Literacy and Numeracy Outcomes in LMICs

Proposal

This talk will provide an overview of the applications of Artificial Intelligence (AI) and Natural Language Processing (NLP) in educational measurement, focusing on automated item generation, automated scoring, and test assembly with a focus on large-scale international assessments such as TIMSS, PIRLS, and others. The talk will describe the unique challenges faced by multilingual assessments when adopting these methods. The talk will give examples of automated scoring of graphical responses in mathematics and science assessments, showing how artificial neural networks can be used to score responses and validate human scorers. In addition, an example of automated scoring of multilingual responses will be provided, which includes a discussion of machine translation. Finally, an example of automated item generation and translation will be provided and discussed in terms of paths to implementation within a human-centered assessment design process.
I will discuss the potential benefits of AI and NLP for educational measurement, including increased efficiency, improved accuracy and reliability of assessment, and increased access to assessment technology for low-resource languages. We will examine the current state of the technology, including challenges associated with developing and deploying AI and NLP-based educational assessment systems. I will also discuss future directions for research and development in this area, including the development of methods for assessing and validating AI- and NLP-based systems and the potential for AI and NLP to improve assessment fairness and reduce assessment bias.

Author

• Matthias von Davier, TIMSS & PIRLS ISC/Boston College