Communication with Surprise – Computational Principles of Goal Signaling in Novel Human Interactions

Tatia Buidze, Tobias Sommer, Ke Zhao, Xiaolan Fu, Jan GläscherbioRxiv 2024 · DOI

TL;DR

This paper introduces a computational model that explains how humans use "surprise" – deviations from expected patterns – to communicate effectively in novel situations without a shared language. The model accurately predicts human communication strategies and is supported by physiological (pupil dilation) and neural (EEG) data, suggesting surprise is a fundamental mechanism for conveying information and guiding attention.

What Problem Does This Paper Address?

Effective communication typically relies on shared language and predictable patterns. However, in novel interactions or when language barriers exist, humans need alternative strategies. This paper addresses how people establish new communication systems from scratch, specifically by leveraging fundamental principles of the physical world and intentionally violating expectations to convey information. The challenge is to understand the computational underpinnings of this "surprise-based" communication and its neural correlates.

Methods

The authors developed a "Surprise model" that simulates communication in a novel, non-verbal game called the Tacit Communication Game (TCG). The model uses two types of "priors":

  • Movement priors: Based on physics (e.g., objects in motion tend to stay in motion), predicting expected movement trajectories.
  • State priors: Representing the sender's goal, guiding the overall direction of communication.

The model generates messages by intentionally violating these priors, creating "information-theoretic surprise." This surprise is the core signal used to convey the receiver's goal. The model also incorporates a reward component to encourage efficient message delivery.

To validate the model, they conducted experiments where human participants played the TCG. They collected behavioral data (message types, profiles), pupillary dilation responses (PDR), and EEG data. The Surprise model's predictions were compared against human behavior and physiological/neural responses.

Datasets

The study used behavioral, PDR, and EEG data collected from two independent samples of participants playing the Tacit Communication Game (TCG).

  • Dataset 1: 58 participants (29 pairs), with 66% indirect and 34% direct trials.
  • Dataset 2: 62 participants (31 pairs), with 76% indirect and 24% direct trials. EEG hyperscanning data (simultaneous EEG recording from both players) was exclusively collected in Dataset 2. The TCG itself is a newly designed experimental task to elicit novel communicative strategies.

Key Insights and Ideas

  • Surprise as a communicative signal: The core idea is that intentionally violating expectations (creating surprise) is a powerful mechanism for conveying information, especially in novel communication scenarios.
  • Leveraging universal priors: Humans revert to fundamental physical principles (like movement kinetics) to establish shared expectations, which can then be strategically violated.
  • Two-phase message planning: The model suggests a two-phase approach: first, conveying the receiver's goal through surprise, and second, efficiently moving towards the sender's goal.
  • Physiological and neural correlates of surprise: Pupillary dilation (PDR) and specific brain activity (frontal/fronto-central EEG) are directly linked to the model's calculated surprise, indicating that surprise is not just a computational concept but has measurable biological effects.
  • Adaptability of human communication: The study highlights how humans can rapidly develop new communication strategies when faced with a lack of shared language, demonstrating remarkable cognitive flexibility.

Key Results

  • The Surprise model accurately predicted human communication strategies in the TCG, outperforming simpler models (Movement-only or State-only). Model comparison showed strong support for the Surprise model across both datasets.
  • Messages generated by the Surprise model closely resembled human-created messages in terms of types and profiles.
  • A significant positive correlation was found between model-derived surprise values and Receiver's pupillary dilation responses (PDR), indicating increased cognitive effort or arousal.
  • EEG analysis revealed distinct time-space clusters in frontal and fronto-central brain regions that correlated significantly with surprise. Source localization pointed to the supplementary motor area (SMA) and dorsal anterior cingulate cortex (dACC) as key areas involved in processing surprising communicative signals.
  • Different message types (e.g., Enter-Exit, Wiggly) were associated with varying levels of surprise, and this variation was reflected in the strength of the EEG signal correlation.

Limitations and Future Work

  • Ecological Validity: The TCG is a controlled, non-verbal setting. Future work could explore more complex, real-world communication scenarios involving a mix of verbal and non-verbal cues, as well as social and emotional contexts.
  • Verbal Domain Translation: Translating the TCG from a non-verbal to a verbal domain would increase ecological validity.
  • Cooperative vs. Competitive Behavior: Exploring how surprise plays a role in cooperative versus competitive social contexts could provide further insights.
  • Emergent Communication: Investigating if communication intentions can naturally emerge without explicit instructions, and how surprise contributes to this emergence, is a promising direction.
  • AI Applications: The principles of surprise-based communication could be applied to AI technologies to create more engaging, creative, and human-like interactions, particularly in areas like generative AI for storytelling or art.

Summary updated

Abstract

Abstract Communication, often grounded in shared expectations, faces challenges without common linguistic backgrounds. Our study explores how people instinctively turn to the fundamental principles of the physical world to overcome communication barriers. Specifically, through the Tacit Communication Game, we investigate how participants develop novel strategies for conveying messages without relying on common linguistic signals. We developed a new computational model built from the principle of expectancy violations of a set of common universal priors derived from movement kinetics. The model closely resembles the Sender’s messages, with its core variable – the information-theoretic surprise – explaining the Receiver’s physiological and neural responses. This is evidenced by a significant correlation with the pupil diameter, indicating cognitive effort, and neural activity in brain areas related to expectancy violations. This work highlights the adaptability of human communication, showing how surprise can be a powerful tool in forming new communicative strategies without relying on common language.
BibTeX
@misc{buidze2024communication,
  author = {Tatia Buidze and Tobias Sommer and Ke Zhao and Xiaolan Fu and Jan Gläscher},
  title = {{Communication with Surprise – Computational Principles of Goal Signaling in Novel Human Interactions}},
  year = {2024},
  doi = {10.1101/2024.02.20.581193},
  howpublished = {bioRxiv},
}
Figure