Focus on What's Informative and Ignore What's not: Communication Strategies in a Referential Game

Roberto Dessì, Diane Bouchacourt, Davide Crepaldi, Marco Baroni

Abstract

This paper investigates how artificial agents develop communication protocols in a referential game. By employing information-theoretic analysis, we demonstrate that agents adapt to the statistical properties of their environment. In non-uniform settings, agents learn to ignore uninformative features and increase the mutual information between messages and informative features. This suggests that environmental skewness can act as a catalyst for more efficient and structured emergent communication.

Introduction

In the field of multi-agent reinforcement learning, agents often develop emergent languages to solve cooperative tasks. However, these languages frequently lack human-like properties such as compositionality. A fundamental question in understanding these systems is the object-symbol mapping: how do agents decide which features of an object (e.g., color, shape, size) are worth communicating? This research explores how the statistical distribution of the environment influences this choice, specifically testing whether agents can learn to ignore uninformative, "noisy" features.

Methods

The researchers utilized a Referential Game involving two neural network agents: a Sender and a Receiver. The Sender observes a target object—represented as a 5-dimensional discrete vector—and generates a message of discrete symbols. The Receiver must then identify the target object from a pair consisting of the target and a random distractor.

Architecture and Analysis

Both agents were implemented using Vanilla Recurrent Neural Networks (RNNs). To enable end-to-end training via backpropagation with discrete symbols, the Gumbel-Softmax trick was employed. The primary tool for analysis was Mutual Information (MI), which quantifies the consistency of the mapping between specific object features and the resulting message symbols. High MI indicates that agents are consistently using specific symbols to describe a particular feature.

Experimental Setup

The study compared two distinct synthetic environments, each containing 1,024 possible objects:

  1. Uniform Environment (U): Each of the five features has four possible values, each appearing with a 25% probability.
  2. Skewed Environment (S): One feature is biased, with one value appearing 75% of the time, making it significantly less useful for discrimination. The remaining four features remain uniform.

Agents were trained on 128,000 pairs and evaluated on their ability to generalize to 4,000 unseen pairs.

Results and Key Insights

Strategic Adaptation to Informativeness

The results demonstrate that agents are highly sensitive to environmental statistics. In the uniform environment, agents typically utilized a random subset of features (usually 3 out of 5) to achieve high accuracy. However, when introduced to the skewed environment, a clear "Aha!" moment occurred: agents recognized the biased feature as a poor discriminator and effectively ignored it.

Table 1: Performance metrics and Mutual Information (MI) across Uniform (U) and Skewed (S) environments. The data shows high accuracy in both cases, but a significant shift in feature prioritization in the skewed environment.

As shown in Table 1, the Mutual Information for the uninformative feature in the skewed environment dropped to nearly zero (0.07). Crucially, by ignoring the useless feature, the agents became significantly better at communicating the remaining informative features. The MI for these features increased by approximately 30% (from ~0.61 to ~0.84) compared to the uniform baseline.

Efficiency and Performance

Despite the change in strategy, both environments maintained high success rates (98.61% for uniform vs. 98.52% for skewed). Agents in the skewed environment developed a more efficient protocol, using slightly fewer unique messages (46 vs. 50) while maximizing the information conveyed about the relevant dimensions.

Discussion and Future Work

These findings suggest that perfectly balanced, "clean" datasets may actually hinder the development of structured emergent languages. More "natural" environments—where some features are common and others are rare—may be necessary to drive agents toward more human-like, efficient communication strategies.

Future research will explore these dynamics in richer environments with hierarchical features and variable-length messages. Additionally, further investigation is needed to determine if this strategic adaptation leads to higher levels of compositionality in the emergent protocols.

Summary updated

Abstract

Research in multi-agent cooperation has shown that artificial agents are able to learn to play a simple referential game while developing a shared lexicon. This lexicon is not easy to analyze, as it does not show many properties of a natural language. In a simple referential game with two neural network-based agents, we analyze the object-symbol mapping trying to understand what kind of strategy was used to develop the emergent language. We see that, when the environment is uniformly distributed, the agents rely on a random subset of features to describe the objects. When we modify the objects making one feature non-uniformly distributed,the agents realize it is less informative and start to ignore it, and, surprisingly, they make a better use of the remaining features. This interesting result suggests that more natural, less uniformly distributed environments might aid in spurring the emergence of better-behaved languages.
BibTeX
@misc{dessi2019focus,
  author = {Roberto Dessì and Diane Bouchacourt and Davide Crepaldi and Marco Baroni},
  title = {{Focus on What's Informative and Ignore What's not: Communication Strategies in a Referential Game}},
  year = {2019},
  eprint = {1911.01892},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.01892},
}

Loading related content...

Figure