Mastering the Tools: Evaluating Machine Learning Tools for Screening in Educational Systematic Reviews (Poster 33)

• In Event: Thursday Poster Session 10:50 am
  In Poster Session: Research Syntheses and Meta-Analyses in Education: Exemplar Studies and Methodological Guidance

Abstract

Title and abstract screening is one of the most time-consuming tasks in a systematic review. Recently, machine learning (ML) tools have been developed to expedite the process, prioritizing the most relevant records to be screened first. The objective of this study is to explore the performance of ML tools used in educational reviews designed to this scope. This paper presents preliminary results on two tools. We conducted screening simulations on ASReview and Covidence with data from two educational systematic reviews. We calculated the proportion of relevant records identified and final full texts included after screening 25%, 50%, and 75% of the studies. Preliminary results showed that the tools present differences in terms of features and performance.

Authors

• Marta Pellegrini, University of Cagliari

• Federico Ferraccioli, University of Padova

• Qiyang Zhang, Johns Hopkins University

• Francesco Marsili, Università degli studi di Perugia

• Livio Finos, University of Padova

• Amanda Neitzel, Johns Hopkins University