Online Multi-Team Systems: Scaling Up Laboratory Experiments to Online Environments

• In Event: Online Experiments: Methods, Opportunities, and Challenges

Abstract

There are a number of benefits for communication researchers to conduct large group experiments in online environments rather than in more traditional offline brick and mortar contexts with college students. While offline group experiments allow a great degree of control over the subjects and provide more face-to-face interaction, online group experiments offer greater scalability, more diverse participant pool, increased potential for replication and/or new manipulations, and significantly reduced costs. In this presentation, we describe our experiences developing and conducting an online version of a fairly complex team simulation that we have been conducting in offline contexts.
Over the past four years, we have collected data from a traditional brick and mortar laboratory experiment known as the Multi-Team System Simulation (MTSS). Twenty participants, organized into four teams of five, participate in a computer-based simulation to move a convoy carrying humanitarian aid through regions that have improvised explosive devices (IEDs) and insurgent activity. The teams within the multiteam system (MTS) share a common distal goal of exchanging any information they might have with other teams in the MTS in order to identify and neutralize IEDs and insurgent activity and ensure safe passage of the convoy. However, each team, which is responsible for a different region, has the more proximate goal of investing their time and effort policing their own region We have used MTSS to conduct over 100 sessions to test the effects of various manipulations on processes and performance by teams as well as the MTS as a whole. These include manipulating communication channels, proximity, team identification, and distribution of information within and between teams in the MTS. We have used relational events analysis to identify a number of patterns of within and between team communication interactions that are more likely to “form” as well as those patterns that are, in some cases, less likely to form but, when they do, are more likely to “perform.”
While these findings are encouraging they face the challenges of other brick and mortar experiments outlined above. Without the spatial limitations of a physical lab, we can increase the number of participants and size of teams in a given simulation. We can obtain a more geographically diverse sample as participants can join the experiment from different locations. Finally, since a physical lab with high-tech equipment is no longer necessary, replication and extensions with new manipulations to our current experiment is more feasible due to the reduced costs. In response we are developing an online MTSS (OMTSS) that will allow us to scale up collection of data to 1000s of sessions, across 100s of different manipulations, with participants from around the globe at a fraction of the cost. Along the way we have had to address several issues including issues such as participant recruitment, training, engagement and retention. This paper will present the issues we faced, the strategies we used to address them as well as experiences from the “field.”

Authors

• Hannah Aubry, Northwestern U

• Eric Forbush, Northwestern University

• William Stawicki, Northwestern U

• Noshir S. Contractor, Northwestern U