Using Artificial Intelligence for Efficient Screening and Coding for Meta-Analyses

• In Event: Leveraging Artificial Intelligence and Automation Tools for Systematic Reviewing

Abstract

This study investigates the use of AI to support large-scale meta-analyses in two of the most labor-intensive and time-consuming phases: screening and coding of full-text articles. Using data from an NSF-funded meta-analysis, we evaluated ChatGPT's accuracy in applying complex eligibility criteria during full-text screening and coding nuanced study characteristics. Our results demonstrate that ChatGPT can apply complex eligibility criteria and screen full-text articles with high accuracy. Additionally, ChatGPT has proven to be a valuable tool in extracting nuanced and multi-layered codes. These findings provide encouraging evidence for integrating AI into the screening and coding processes of meta-analyses, highlighting its potential to enhance efficiency and accuracy.

Authors

• Qi Zhang, American Institutes for Research

• Sarah Sahni, J-PAL North America, MIT

• Megha Joshi, American Institutes for Research

• Martyna Citkowicz, American Institutes for Research

• Kamal Middlebrook, American Institutes for Research

• Laura Michaelson, American Institutes for Research