Exploring Generative Artificial Intelligence for Graphical Data Extraction in Single Case Design Meta-analysis

• In Event: Generative AI and Large Language Models in Educational Measurement and Assessment

Abstract

Meta-analysis is a common approach in single-case experimental design (SCED) research to synthesize evidence and identify broader patterns of intervention effectiveness. Researchers must extract data from both text and graphs. However, graphical data extraction is often time-consuming and labor-intensive. This study evaluates the performance of three widely used generative AI tools (i.e., ChatGPT, Gemini, and Claude) in automating graphical data extraction. The study also examines whether the accuracy of generative AI varies by graph characteristics (e.g., the number of baseline/intervention data points, trend, variability). The findings provide practical guidance on selecting optimal AI tools to streamline graphical data extraction in SCED meta-analyses. Additionally, the study identifies the specific conditions under which generative AI tools are most effective.

Authors

• Yaosheng Lou, University at Albany - SUNY

• Mariola Moeyaert, University at Albany - SUNY

• Piet Wesling, University at Albany - SUNY

• Benjamin G. Solomon, University at Albany - SUNY

• Xin Li, University at Albany - SUNY