A Computational Method for Scoring Parent Expressed Emotion in ADHD and Other Child Conditions

• In Event: Poster Session 10
  In Poster Session: PS 10 Topic - Methods, History, Theory

Abstract

Introduction: This study evaluates validity of sentiment analysis of the Five Minute Speech Sample (FMSS). The FMSS is a technique for measuring that affective climate of the home environment, focusing on how parents describe their relationship with their children following an open prompt. Coding to criterion using the standard EE coding scheme (Amato, 1993) is a skilled, time-consuming task requiring well trained coders. Sentiment Analysis is a computational natural language processing technique to classify speech into negative, positive, or neutral content. Sentiment analysis could reduce FMSS data coding burden substantially.

Methods: We applied sentiment analysis and the validated expressed emotion (EE) (Malla et al., 1991) coding system to the same 599 parent (82% mothers) FMSS transcripts. Their children were aged 7-13, (M=9.11, SD=1.48; 62% male). Detailed research evaluation identified ADHD (62% of the children) and non-ADHD cases (Nigg et al., 2018). Transcripts were first coded by the EE coding system, by standard methods by validated and reliable coders (blind to the sentiment analysis output) as outlined by Amato (1993). Frequency of parental criticism and positive remarks served as the primary outcome variables for this report. Sentiment analysis was then implemented on the same transcripts (blind to the EE coding) using the Valence Aware Dictionary for sEntiment Reasoning (VADER) model (Hutto & Gilbert, 2014). VADER uses a dictionary that maps phrases to emotion intensities. It has 96% classification accuracy for assessing positive and negative sentiment of short phrases or sentences (Hutto & Gilbert, 2014). FMSS transcripts were analysed at the sentence-level using VADER (n=19,565 sentences). The average score for each transcript provided the positive and negative sentiment variables for analysis.

Results: Convergent validity for VADER scores with conceptually related EE scores was encouraging (Figure 1). As shown, VADER negative sentiment correlated with EE criticism, r(597) = .39, (p = 2.2e-16) and was negatively correlated with EE positive remarks, r(597) = -.24, (p = 4.5e-09). Likewise, VADER positive sentiment was correlated with EE positive remarks r(597) = .33, (p = 2.4e-16) and negatively correlated with EE criticism r(597) = -.23, (p =1.9e-08). Additionally, VADER sentiment scores and EE scores were similarly correlated with standardized rating measures of child psychopathology (see Table 1). Additional regression models, demonstrating the incremental prediction of VADER over EE scores, on child psychopathology measures will be reported as well.

Conclusion: VADER sentiment scores of FMSS appear to adequately capture parent FMSS expressed emotion in a valid and reliable manner at the molar order of positive and negative sentiment. However, VADER and EE coding captured unique aspects of FMSS emotion tone as well. Although not a replacement for EE coding, a computational scoring system could be used to quickly gather emotional tone data from the FMSS and be useful for research in which expert coding is either infeasible or not needed to test hypotheses.

Authors

• Katharine van der Hoorn, Oregon Health & Science University (OHSU)
  Presenting Author

• Natalie Miller, Oregon Health and Sciences University
  Non-Presenting Author

• Sarah Karalunas, Purdue University, West Lafayette
  Non-Presenting Author

• Tara Peris, UCLA
  Non-Presenting Author

• Joel T Nigg, Oregon Health & Science University (OHSU)
  Non-Presenting Author