Emergent Quantized Communication

Boaz Carmeli, Ron Meir, Yonatan Belinkov

Abstract

The field of emergent communication aims to understand the characteristics of communication as it emerges from artificial agents solving tasks that require information exchange. Communication with discrete messages is considered a desired characteristic, for scientific and applied reasons. However, training a multi-agent system with discrete communication is not straightforward, requiring either reinforcement learning algorithms or relaxing the discreteness requirement via a continuous approximation such as the Gumbel-softmax. Both these solutions result in poor performance compared to fully continuous communication. In this work, we propose an alternative approach to achieve discrete communication -- quantization of communicated message. Using message quantization allows us to train the model end-to-end, achieving superior performance in multiple setups. Moreover, quantization is a natural framework that runs the gamut from continuous to discrete communication. Thus, it sets the ground for a broader view of multi-agent communication in the deep learning era.

Introduction

In emergent communication research, the goal is to observe how artificial agents develop protocols to solve cooperative tasks, such as a sender describing an object for a receiver to identify. Ideally, these protocols should utilize discrete symbols—akin to human language—to facilitate interpretability and human-machine interfacing.

However, the standard optimization tool for neural networks, gradient descent, requires continuous and differentiable data. Forcing agents to output discrete symbols typically breaks the gradient flow. To circumvent this, researchers have historically relied on Reinforcement Learning (REINFORCE) or continuous approximations like Gumbel-softmax. Unfortunately, these methods often suffer from a performance gap: as the communication channel becomes more discrete, agent performance tends to degrade significantly compared to continuous vector-based communication. This paper introduces message quantization as a robust alternative that bridges the gap between discrete requirements and continuous optimization.

Methodology: Message Quantization

Rather than using stochastic sampling, the proposed method utilizes deterministic Quantization borrowed from neural network compression. In this framework, the sender generates a continuous message vector which is then mapped to discrete, uniformly spaced integer bins (symbols) before reaching the receiver.

Comparison of message representations and accuracy across communication types.

The Forward and Backward Pass

During the Forward Pass, continuous values are scaled, rounded to the nearest integer, and then scaled back, effectively creating a discrete alphabet. To handle the non-differentiable nature of the rounding operation during the Backward Pass, the authors employ the Straight-Through Estimator (STE). This treats the rounding step as an identity function for gradients, allowing them to flow back to the sender network unimpeded.

Channel capacity is controlled via two primary parameters: alphabet size (number of bins) and word length (dimension of the message vector). This approach allows for end-to-end training while maintaining a strictly discrete bottleneck at inference time.

Experimental Setup

The effectiveness of quantized communication was evaluated across four distinct cooperative games:

  • Synthetic Objects: 10,000 objects defined by discrete attributes.
  • Images: A subset of ImageNet (463 classes) requiring visual feature extraction.
  • Sent-Ref (Text): Identifying specific customer support queries from the Banking77 dataset.
  • Sent-Cls (Text): Identifying the intent category of customer support queries.

Key Results and Discussion

The experimental results demonstrate that quantized communication is a highly effective mechanism for emergent language, consistently outperforming stochastic approximations.

Accuracy across four different games and data types, comparing continuous (CN), Gumbel-softmax (GS), and quantized (QT) methods.

Performance Parity with Continuous Channels

Across all games, quantized communication achieved accuracy levels virtually on par with fully continuous communication. Even as the number of distractors (candidate objects) increased to 10,000, the quantized method maintained high performance (>92% in most tasks), whereas Gumbel-softmax performance collapsed.

Capacity Utilization

A critical insight is that Gumbel-softmax often fails because it cannot effectively utilize the available channel capacity. In the synthetic object task, Gumbel-softmax generated only 58 unique messages for 1,000 targets, leading to failure. In contrast, the quantized channel successfully generated 1,000 unique messages, achieving perfect accuracy. Furthermore, the results suggest that word length is more critical than alphabet size; agents can communicate perfectly even with a binary alphabet (size 2) if the message vector is sufficiently long.

Conclusion

Quantized communication offers a robust and stable alternative to traditional discrete communication methods in multi-agent systems. By leveraging the Straight-Through Estimator, it allows agents to learn discrete protocols that match the performance of continuous ones. While this study utilized uniform quantization, future work may explore non-uniform schemes or more complex architectures to further enhance the efficiency and human-readability of emergent artificial languages.

Summary updated

Abstract

The field of emergent communication aims to understand the characteristics of communication as it emerges from artificial agents solving tasks that require information exchange. Communication with discrete messages is considered a desired characteristic, for scientific and applied reasons. However, training a multi-agent system with discrete communication is not straightforward, requiring either reinforcement learning algorithms or relaxing the discreteness requirement via a continuous approximation such as the Gumbel-softmax. Both these solutions result in poor performance compared to fully continuous communication. In this work, we propose an alternative approach to achieve discrete communication -- quantization of communicated message. Using message quantization allows us to train the model end-to-end, achieving superior performance in multiple setups. Moreover, quantization is a natural framework that runs the gamut from continuous to discrete communication. Thus, it sets the ground for a broader view of multi-agent communication in the deep learning era.
BibTeX
@inproceedings{carmeli2023emergent,
  author = {Boaz Carmeli and Ron Meir and Yonatan Belinkov},
  title = {{Emergent Quantized Communication}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=TPe10Uh92r},
}
Figure