Conversation dynamics in a multiplayer video game with knowledge asymmetry

James Simpson, Patrick Nalepka, Rachel W. Kallen, Mark Dras, Erik D. Reichle, Simon G. Hosking, Christopher Best, Deborah Richards, Michael J. RichardsonFrontiers in Psychology 2022 · DOI

TL;DR

This study uses a custom multiplayer video game, "Desert Herding," to investigate how communication dynamics change in teams with asymmetric knowledge, particularly when visual information is degraded. They found that reduced visibility forces teams to reorganize their communication, with the drone operator taking a more central role, and that this reorganization, while increasing communication, can sometimes lead to worse performance.

What Problem Does This Paper Address?

Remote collaboration is increasingly common, especially in online gaming communities. However, the challenges of virtual communication, particularly the lack of non-verbal cues, can hinder effective teamwork. This paper addresses how teams maintain coordination and "team situation awareness" (TSA) in a complex, cooperative task with inherent knowledge asymmetry between players. Specifically, they explore how manipulating the visual environment (introducing "fog") affects communication patterns and team performance, drawing parallels to human-AI teaming scenarios.

Methods

The researchers developed a custom cooperative multiplayer video game called "Desert Herding." In this game, four-player teams consist of three "ground players" who navigate a desert environment to herd robot agents, and one "drone operator" who has a bird's-eye view and access to a map with complete information about agent and player locations.

To analyze communication, they employed Categorical Recurrence Quantification Analysis (catRQA). This technique quantifies the dynamics of communication by analyzing patterns in speech over time. Specifically, they measured:

  • %REC (Percent Recurrent Points): Indicates how often the same communication states recur.
  • %DET (Percent Determinism): Measures how structured or deterministic communication patterns are.

Task difficulty was manipulated by introducing environmental "fog," which reduced ground players' visibility but not the drone operator's access to information. This created a knowledge asymmetry that was amplified by the fog.

Datasets

The study did not use pre-existing datasets. Instead, they collected data from ten four-person teams playing their custom-developed game, "Desert Herding." The data analyzed was specifically from the fourth and final session of each team's participation, intended to capture skilled-level performance.

Key Insights and Ideas

  • Fog as a Catalyst for Communication Reorganization: Reduced visibility (fog) significantly increased overall team communication and made it more structured (%REC and %DET). This suggests that when information is scarce, teams rely more heavily on verbal communication to maintain shared understanding.
  • The Drone Operator as a Central Hub: The drone operator's communication became more frequent and deterministic under foggy conditions. This highlights their crucial role in compensating for the ground players' limited perception and acting as a central information conduit.
  • Ground Player Communication Suffers Under Fog: Counterintuitively, while overall team communication increased, the ground players' communication with each other became less recurrent (%REC decreased) in foggy conditions. This suggests a shift in focus, potentially towards communicating with the operator, or a breakdown in peer-to-peer coordination.
  • Communication Structure Can Hinder Performance: Interestingly, more recurrent and deterministic communication among ground players was associated with worse task performance (longer trial durations, lower containment rates). This implies that while structured communication might seem efficient, it can become a bottleneck if it's not effectively conveying the necessary information or if it leads to rigid, less adaptable strategies.
  • Team-Level Communication Structure Can Aid Performance: In contrast to ground player communication, more recurrent communication at the team level was associated with better performance (faster completion times). This suggests that a more synchronized and predictable communication flow across the entire team, possibly driven by the operator's role, can be beneficial.

Key Results

  • Team Performance: Teams completed trials faster and contained targets at a higher rate when there were fewer targets (9 vs. 18), visibility was high (no fog), and no task perturbation occurred. Fog significantly increased trial duration and decreased containment rate.
  • Team Division of Labor: Teams divided labor more efficiently (less overlap in search areas) when visibility was high and when there was no task perturbation.
  • Magnitude of Verbal Communication: Both the drone operator and ground players spoke more when fog was present. The drone operator's speech increased significantly with fog, and ground players also spoke more when fog was present and when more targets needed corralling.
  • Structural Dynamics of Verbal Communication:
    • Team Level: %REC and %DET were higher with fog, indicating more recurrent and deterministic team communication.
  • Operator: %REC and %DET were significantly higher with fog, showing more frequent and structured communication from the operator.
  • Ground Players: %REC was significantly lower with fog, indicating less recurrent communication among ground players.
  • Relationship between Communication and Performance:
    • Increased magnitude of ground player communication was linked to worse performance (longer trial duration, lower containment rate).
  • More recurrent ground player communication (%REC) was also associated with worse task performance.
  • More recurrent team-level communication (%REC) was associated with better performance.

Limitations and Future Work

  • Focus on Skilled Performance: The study primarily analyzed data from the fourth session, assuming participants had reached a skilled level. This might limit generalizability to teams in earlier stages of learning.
  • Custom Game Environment: While "Desert Herding" was designed to simulate real-world challenges, it is still a simplified environment. Findings may not directly translate to all complex collaborative tasks.
  • Artificial Agents for Teaming: The authors highlight the potential for artificial agents to act as synthetic teammates, particularly in the drone operator role. Future work could explore how AI agents can be designed to effectively communicate and coordinate with human teams, potentially mitigating the challenges observed with fog. They suggest that agents need to monitor conversations, anticipate needs, and engage in more natural, dyadic interactions than current platforms typically allow.
  • Mitigating Fog's Impact: The paper references prior work suggesting that providing ground players with a Heads-Up Display (HUD) showing the map could mitigate the negative impacts of fog on team performance, indicating a direction for future game design and human-AI interaction research.

Summary updated

Abstract

Despite the challenges associated with virtually mediated communication, remote collaboration is a defining characteristic of online multiplayer gaming communities. Inspired by the teamwork exhibited by players in first-person shooter games, this study investigated the verbal and behavioral coordination of four-player teams playing a cooperative online video game. The game, <i>Desert Herding</i>, involved teams consisting of three <i>ground players</i> and one <i>drone operator</i> tasked to locate, corral, and contain evasive robot agents scattered across a large desert environment. Ground players could move throughout the environment, while the drone operator's role was akin to that of a "spectator" with a bird's-eye view, with access to veridical information of the locations of teammates and the to-be-corralled agents. Categorical recurrence quantification analysis (catRQA) was used to measure the communication dynamics of teams as they completed the task. Demands on coordination were manipulated by varying the ground players' ability to observe the environment with the use of game "fog." Results show that catRQA was sensitive to changes to task visibility, with reductions in task visibility reorganizing how participants conversed during the game to maintain team situation awareness. The results are discussed in the context of future work that can address how team coordination can be augmented with the inclusion of artificial agents, as synthetic teammates.
BibTeX
@inproceedings{simpson2022conversation,
  author = {James Simpson and Patrick Nalepka and Rachel W. Kallen and Mark Dras and Erik D. Reichle and Simon G. Hosking and Christopher Best and Deborah Richards and Michael J. Richardson},
  title = {{Conversation dynamics in a multiplayer video game with knowledge asymmetry}},
  booktitle = {Frontiers in Psychology},
  year = {2022},
  doi = {10.3389/fpsyg.2022.1039431},
}
Figure