Abstract
This paper investigates how artificial agents develop communication protocols in a referential game. By employing information-theoretic analysis, we demonstrate that agents adapt to the statistical properties of their environment. In non-uniform settings, agents learn to ignore uninformative features and increase the mutual information between messages and informative features. This suggests that environmental skewness can act as a catalyst for more efficient and structured emergent communication.
Introduction
In the field of multi-agent reinforcement learning, agents often develop emergent languages to solve cooperative tasks. However, these languages frequently lack human-like properties such as compositionality. A fundamental question in understanding these systems is the object-symbol mapping: how do agents decide which features of an object (e.g., color, shape, size) are worth communicating? This research explores how the statistical distribution of the environment influences this choice, specifically testing whether agents can learn to ignore uninformative, "noisy" features.
Methods
The researchers utilized a Referential Game involving two neural network agents: a Sender and a Receiver. The Sender observes a target object—represented as a 5-dimensional discrete vector—and generates a message of discrete symbols. The Receiver must then identify the target object from a pair consisting of the target and a random distractor.
Architecture and Analysis
Both agents were implemented using Vanilla Recurrent Neural Networks (RNNs). To enable end-to-end training via backpropagation with discrete symbols, the Gumbel-Softmax trick was employed. The primary tool for analysis was Mutual Information (MI), which quantifies the consistency of the mapping between specific object features and the resulting message symbols. High MI indicates that agents are consistently using specific symbols to describe a particular feature.
Experimental Setup
The study compared two distinct synthetic environments, each containing 1,024 possible objects:
- Uniform Environment (U): Each of the five features has four possible values, each appearing with a 25% probability.
- Skewed Environment (S): One feature is biased, with one value appearing 75% of the time, making it significantly less useful for discrimination. The remaining four features remain uniform.
Agents were trained on 128,000 pairs and evaluated on their ability to generalize to 4,000 unseen pairs.
Results and Key Insights
Strategic Adaptation to Informativeness
The results demonstrate that agents are highly sensitive to environmental statistics. In the uniform environment, agents typically utilized a random subset of features (usually 3 out of 5) to achieve high accuracy. However, when introduced to the skewed environment, a clear "Aha!" moment occurred: agents recognized the biased feature as a poor discriminator and effectively ignored it.
As shown in Table 1, the Mutual Information for the uninformative feature in the skewed environment dropped to nearly zero (0.07). Crucially, by ignoring the useless feature, the agents became significantly better at communicating the remaining informative features. The MI for these features increased by approximately 30% (from ~0.61 to ~0.84) compared to the uniform baseline.
Efficiency and Performance
Despite the change in strategy, both environments maintained high success rates (98.61% for uniform vs. 98.52% for skewed). Agents in the skewed environment developed a more efficient protocol, using slightly fewer unique messages (46 vs. 50) while maximizing the information conveyed about the relevant dimensions.
Discussion and Future Work
These findings suggest that perfectly balanced, "clean" datasets may actually hinder the development of structured emergent languages. More "natural" environments—where some features are common and others are rare—may be necessary to drive agents toward more human-like, efficient communication strategies.
Future research will explore these dynamics in richer environments with hierarchical features and variable-length messages. Additionally, further investigation is needed to determine if this strategic adaptation leads to higher levels of compositionality in the emergent protocols.
Summary updated