The Proficiency-Congruency Dilemma: Virtual Team Design and Performance in Multiplayer Online Games

• In Event: Drowning in Data: Industry and Academic Approaches to Mixed Methods in “Holistic” Big Data Studies

Abstract

I have been introduced to a mixed-methods approach to data analysis by a social scientist; I believe many computer scientists are in the same situation. Most of the research in my field of machine learning does not consider personal interviews or focus groups as a relevant part of the story. However, analyzing the results of a quantitative experiment often leaves important questions unanswered which cannot be always resolved with more quantitative studies. I, like many of my colleagues, have been looking toward interdisciplinary research where I can learn from social scientists about this mixed-methods approach. Here I present one of my first attempts at interdisciplinary, mixed-methods research.

Multiplayer online battle arena games provide an excellent opportunity to study team performance. When designing a team, players must negotiate a proficiency-congruency dilemma between selecting roles that best match their experience and roles that best complement the existing roles on the team. We adopt a mixed-methods approach to explore how users negotiate this dilemma. Using data from League of Legends, we define a champion similarity space to operationalize constructs about proficiency, generality, and congruency. We collect publicly-available data from approximately 3.36 million users to test the influence of these constructs on team performance. We also conduct focus groups with novice and elite players to understand how players' team design practices vary with expertise, synthesized with the findings from the machine learning analysis. We find support for hypotheses that player proficiency and team congruency increase team performance. These findings have implications for players, designers, and theorists about how to recommend team designs that jointly prioritize individuals' expertise and teams' compatibility.

Author

• Alice Oh, KAIST