Beyond the Message: The Impact of Visual and Audio Presentation Styles on Online Engagement

• In Event: Communication, Information Technologies, and Media Sociology Section Roundtables
  In Refereed Roundtable (60 minute): 6. Presentation, propaganda, & politicization

Abstract

How do political actors generate social influence? Conventional research largely centers the role of text/language as the vehicle of persuasion. Yet, such an emphasis no longer captures the full picture of social media campaigning, where candidates communicate through videos that present visual and audio cues alongside text. We ask the question of “multimodal influence”: how do candidates’ nonverbal self-presentation, as reflected in audio and visual cues, shape online engagement beyond the linguistic content of their messages?

We analyze Instagram videos by 12 Senate candidates in the 2024 U.S. elections. Leveraging advances in computational tools that group audio and visual cues, we categorize self-presentation into multimodal styles. We identify clusters of audio features (e.g., pitch variability, speaking rate, intonation) and facial expression dynamics that correspond to distinct styles. We then examine how these styles relate to online engagement, specifically the number of likes and comments.

Across models that control for linguistic topics and include candidate fixed effects, we find that nonverbal styles are systematically associated with engagement. A resonant audio style (i.e., one with a more authoritative and dynamic vocal delivery) is associated with higher levels of both likes and comments. In contrast, a low-key audio style is negatively associated with commenting behavior. Surprisingly, an animated audio style that emphasizes inflammatory speech is not associated with online engagement. For visual presentation, a measured style is positively associated with comments, whereas expressive or restrained visual styles are not linked to greater engagement.

Taken together, these findings suggest that online engagement is shaped not only by what candidates say, but by how they sound and appear when saying it. More broadly, the study highlights the importance of conceptualizing social influence on social media as multimodal and style-based, and demonstrates how computational approaches can be used to analyze embodied self-presentation at scale.

Authors

• Yuan Hsiao, Yale University

• Ching-Ching Chang, Academia Sinica