Gender Bias in Online Content Moderation and Its Consequences for Online Discourse

• In Event: 20220 - Section on Communication, Information Technology, and Media Sociology Roundtables
  In Refereed Roundtable (60 minute): Section on Communication, Information Technology, and Media Sociology Roundtable IV: Digital Infrastructure II

Abstract

Online toxicity shapes user participation and digital inclusion, yet the role of moderator biases—particularly those related to gender—remains poorly understood. This study examines how gender differences in toxicity perception and content moderation shape men's and women's experiences on social media through a novel simulation framework informed by large-scale annotation surveys. Using data from 17,280 participants who evaluated 107,260 online comments, we discover a crucial gender asymmetry in moderation: men are less likely to moderate comments flagged as toxic by women, while both genders moderate comments identified as toxic by men at similar rates. To assess the impact of moderator demographics, we develop a counterfactual simulation that models content moderation outcomes based on different moderator team compositions. Results reveal a clear “in-group bias”: male-majority teams reduce perceived toxicity for men but not women, whereas female-majority teams have the opposite effect. When simulating a Reddit-like moderation team, we find that post-moderation content would remain more toxic for women. However, a moderation team reflecting national demographics could more effectively remove content perceived as toxic by women, making the online environment less toxic for women. These findings highlight the real-world consequences of biased moderation, showing that a lack of gender diversity in moderation teams can make online spaces more hostile, discourage women's participation, and reinforce online gender inequalities.

Authors

• Zhaodi Chen, Indiana University-Bloomington

• Byungkyu Lee, New York University