Individual Submission Summary

Direct link:

Intervening “Just-in-Time”: Supervised Machine Learning to Detect Recovery Problems in an Online Support Forum

Sat, May 27, 9:30 to 10:45, Hilton San Diego Bayfront, Floor: 4, Sapphire 411 AB


Background: Internet-supported forums allow those in addiction recovery to seek help through text-based messages. Forum moderators can provide valuable support when participants are struggling, but considerable labor is required to review newly posted content. This paper investigates whether advances in natural language processing and machine learning may facilitate automatically identifying concerning messages in real-time, allowing for more timely and efficient engagement by forum moderators.
Methods: For training data, messages from a recovery forum were labeled for whether or not authors mentioned recovery problems. Binary classifiers for recovery problems leveraged supervised machine learning, contrasting several natural language processing approaches: 1) a Bag-of-Words (BoW) model 2) the dictionary-based Linguistic Inquiry and Word Count (LIWC) program and 3) a hybrid approach combining BoW and LIWC.
Results: A Boosted Decision Tree classifier, utilizing features from both BoW and LIWC, achieved 88% sensitivity and 82% specificity in an additional recovery forum.
Conclusions: Incorporating predictive models within health forums could enable moderators to focus their attention, in real-time, on the most concerning content.