Application of Text Mining for Systematic Reviews of Qualitative Research - A Case Study Using Hospital-Based Violence Intervention Programs (HVIPs)

• In Event: Understanding Violent Victimization - Advancing Methods and Theory

Abstract

Systematic reviews are essential for researchers, practitioners and policy makers for rational decision making by efficiently integrating existing information, establishing whether scientific findings are consistent and can be generalized across populations, settings, and treatment variations, or whether findings vary significantly by particular subsets. In an attempt to use established manual procedures to systematically review the qualitative evidence available about HVIPs we were faced with numerous challenges, most important among them being paucity of time. Therefore, we compared the use of computational text mining techniques with manual techniques to see if they saved time and generated similar results for systematic reviews. Using the qualitative literature relating to HVIPs we compared: (i) the manual process involving domain experts who reviewed eighty-one articles in the subject area; and, (ii) the computational process using LDA (Latent Dirichlet Allocation) and LLM (Large Language Model) algorithms to do the same. The purpose of the manual review was to validate the computer's findings and to report measurements such as sensitivity, specificity, recall and accuracy. In both cases, domain experts are essential, but the computational method can save significant amounts of time by automating certain processes. Additionally, the computational approach would make the results repeatable to quickly facilitate incorporation of future studies as they are published to update the findings of the systematic reviews. In this presentation, we compare the results of the systematic review using these two approaches (manual versus computational), identify the steps where time savings can be achieved and where domain expertise is needed to ensure accuracy.

Authors

• Cyril S. Ku, William Paterson University

• Morgan Peltier, Hackensack Meridian Health Network

• Sheetal Ranjan, Montclair State University