A Scoping Review of Machine Learning and Natural Language Processing Applications in Evaluation Studies.

• In Event: Ethical Engagement, STEM Programs, and Application of Machine Learning Techniques in Evaluation Research

Abstract

Machine learning (ML) and natural language processing (NLP) are increasingly being used in evaluation studies (Anglin, 2019; Cintron & Montrosse-Moorhead, 2022; Wencker, et al., 2025). Yet, little is known about the trends in the use of ML and NLP across evaluation studies (Bouyousfi & Ouedraogo, 2024). To describe the state of practice, this scoping review assesses how ML and NLP are applied in evaluation studies, and if and how often validity is addressed within these studies. Preliminary findings have revealed limited detail on the specific ML and NLP methods used in practice, and minimal and often implicit discussions of validity, highlighting the need for more thorough methodological reporting.

Authors

• Ibrahim Osumanu, University of Connecticut

• Bianca Montrosse-Moorhead, University of Connecticut

• Kylie Anglin, University of Connecticut

• Christopher H. Rhoads, University of Connecticut