Emergent Quantized Communication

Boaz Carmeli, Ron Meir, Yonatan Belinkov

Abstract

The field of emergent communication aims to understand the characteristics of communication as it emerges from artificial agents solving tasks that require information exchange. Communication with discrete messages is considered a desired characteristic, for both scientific and applied reasons. However, training a multi-agent system with discrete communication is not straightforward, requiring either reinforcement learning algorithms or relaxing the discreteness requirement via a continuous approximation such as the Gumbel-softmax. Both these solutions result in poor performance compared to fully continuous communication. In this work, we propose an alternative approach to achieve discrete communication -- quantization of communicated messages. Using message quantization allows us to train the model end-to-end, achieving superior performance in multiple setups. Moreover, quantization is a natural framework that runs the gamut from continuous to discrete communication. Thus, it sets the ground for a broader view of multi-agent communication in the deep learning era.

Introduction

In emergent communication, the goal is for artificial agents to develop a shared protocol to solve collaborative tasks. Human language is inherently discrete, composed of distinct symbols and words. Replicating this discreteness in AI agents is desirable for interpretability and efficiency. However, modern deep learning relies on gradient-based optimization, which requires continuous functions. Discrete choices create "step functions" with zero gradients, making standard backpropagation impossible.

Existing solutions like Reinforcement Learning (RL) or Gumbel-softmax (GS) approximations often suffer from instability or significant performance drops—a "discreteness tax"—when compared to agents using unconstrained continuous communication. This paper introduces Quantized Communication (QT) as a robust alternative that bridges the gap between discrete constraints and continuous optimization.

Methodology: Quantized Communication

The core proposal is to apply quantization techniques to the messages exchanged between agents. Instead of selecting a discrete symbol directly (as in GS), the sender generates a continuous vector which is then mapped to a finite set of integers through uniform quantization.

Conceptual comparison of message structures and performance scaling in the Object game.

Key Technical Components

  • Uniform Quantization: Real-valued signals are mapped into a fixed alphabet (e.g., binary {0, 1} or larger sets).
  • Straight-Through Estimator (STE): To allow gradients to flow through the non-differentiable rounding operation, the STE treats the operation as an identity function during the backward pass.
  • Channel Architectures: The authors explored both "Instant" channels (single word transmission) and "Recurrent" channels (sequences of words generated by an RNN).

As shown in the conceptual comparison above, Quantized communication maintains the high accuracy of continuous signals even as the number of candidates increases, whereas Gumbel-softmax performance degrades rapidly.

Experimental Setup

The effectiveness of QT was evaluated across four distinct games:

  1. Object Game: Agents identify an object based on discrete attributes.
  2. Image Game: Agents identify a specific ImageNet image among distractors.
  3. Sent-Ref (Referential): Identifying a specific sentence from the Banking77 dataset.
  4. Sent-Cls (Classification): Identifying the intent class of a sentence.

Results and Discussion

The results demonstrate that Quantized communication effectively eliminates the performance penalty associated with discrete channels.

Performance comparison across four benchmark tasks: Object, Image, Sent_Ref, and Sent_Cls.

Performance and Scaling

In almost every task, QT communication performed on par with the fully continuous (CN) baseline, which serves as the theoretical upper bound. Notably, in the Object game with 1,000 candidates, QT maintained 100% accuracy, while GS plummeted to approximately 44%. This "graceful scaling" suggests that QT is far more robust to task complexity than previous discrete approximations.

Optimization and Message Uniqueness

A key insight is that QT is easier for neural networks to optimize. While GS agents often suffer from "vocabulary collapse"—using only a few unique messages for many different targets—QT agents successfully generate unique messages for nearly every target. For instance, in referential tasks, QT agents produced 1,000 unique messages for 1,000 targets, ensuring perfect disambiguation.

Conclusion

Quantized Communication provides a "best of both worlds" solution for multi-agent systems. It allows for the emergence of discrete, symbol-based protocols without sacrificing the performance and ease of training associated with continuous signals. By leveraging quantization and the Straight-Through Estimator, researchers can develop more interpretable and efficient agent communication protocols that scale to complex, high-dimensional tasks. Future work may explore non-uniform quantization schemes and deeper semantic analysis of the emergent "quantized" languages.

Summary updated

Abstract

The field of emergent communication aims to understand the characteristics of communication as it emerges from artificial agents solving tasks that require information exchange. Communication with discrete messages is considered a desired characteristic, for both scientific and applied reasons. However, training a multi-agent system with discrete communication is not straightforward, requiring either reinforcement learning algorithms or relaxing the discreteness requirement via a continuous approximation such as the Gumbel-softmax. Both these solutions result in poor performance compared to fully continuous communication. In this work, we propose an alternative approach to achieve discrete communication -- quantization of communicated messages. Using message quantization allows us to train the model end-to-end, achieving superior performance in multiple setups. Moreover, quantization is a natural framework that runs the gamut from continuous to discrete communication. Thus, it sets the ground for a broader view of multi-agent communication in the deep learning era.
BibTeX
@inproceedings{carmeli2023emergent,
  author = {Boaz Carmeli and Ron Meir and Yonatan Belinkov},
  title = {{Emergent Quantized Communication}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=TPe10Uh92r},
}
Figure