Unsupervised Translation of Emergent Communication

Ido Levy, Orr Paradise, Boaz Carmeli, Ron Meir, Shafi Goldwasser, Yonatan Belinkov

Abstract

Emergent Communication (EC) provides a unique window into the language systems that emerge autonomously when agents are trained to jointly achieve shared goals. However, it is difficult to interpret EC and evaluate its relationship with natural languages (NL). This study employs unsupervised neural machine translation (UNMT) techniques to decipher ECs formed during referential games with varying task complexities, influenced by the semantic diversity of the environment. Our findings demonstrate UNMT's potential to translate EC, illustrating that task complexity characterized by semantic diversity enhances EC translatability, while higher task complexity with constrained semantic variability exhibits pragmatic EC, which, although challenging to interpret, remains suitable for translation. This research marks the first attempt, to our knowledge, to translate EC without the aid of parallel data.

Introduction: The Challenge of AI Language

In multi-agent reinforcement learning, agents often develop their own communication protocols—known as Emergent Communication (EC)—to achieve shared goals. While these protocols are efficient for the agents, they are notoriously opaque to humans. Historically, researchers have relied on mathematical metrics or supervised translation models requiring parallel data (pairs of AI messages and human descriptions) to interpret them. However, supervised methods can introduce human bias, forcing AI protocols into human linguistic structures. This paper introduces a novel method to translate these "alien" languages into English using Unsupervised Neural Machine Translation (UNMT), requiring no parallel data and allowing for a more objective analysis of AI-generated communication.

Methodology: A Two-Phase Approach

The researchers developed a framework consisting of two main phases: generating the communication and then translating it.

1. Generating Emergent Communication

The study utilizes a "referential game" involving a Sender and a Receiver. The Sender observes a target image and generates a discrete, 6-symbol message. The Receiver must then use this message to identify the target image from a lineup of nine "distractor" images. To understand how environment shapes language, the authors varied task complexity by changing the relationship between the target and distractors:

  • Random: Distractors are unrelated to the target.
  • Category: Distractors are the same object type as the target.
  • Supercategory: Distractors belong to the same broad family (e.g., different animals).
  • Inter-category: Distractors share overlapping features or contexts.

2. Unsupervised Translation (UNMT)

The messages from these games formed an "EC corpus," which was paired with an "English corpus" of human-annotated image captions from the MSCOCO dataset. Using a standard UNMT pipeline, the authors pre-trained a language model (XLM) on English and fine-tuned it on the EC data. Through back-translation and denoising, the model learned to align the two languages without ever seeing a direct translation pair.

The pipeline for generating emergent communication through referential games and translating it using unsupervised machine translation.

Key Insights: Environment and Translatability

The study revealed several counterintuitive findings regarding how agents communicate:

  • Unsupervised Translation is Viable: The UNMT system successfully mapped discrete AI symbols to coherent English sentences, proving that AI-generated protocols share enough structural similarity with human language to be translated without a "Rosetta Stone."
  • Complexity Does Not Equal Translatability: The most difficult visual tasks did not always yield the most translatable languages. The Inter-category games produced the most human-like, translatable language. In contrast, Supercategory games resulted in high-entropy vocabularies that were significantly harder to translate.
  • Pragmatic Efficiency: In highly specific tasks (e.g., distinguishing between two similar giraffes), agents used smaller vocabularies. They optimized for efficiency, likely communicating low-level visual features (like "white background") rather than high-level abstract concepts.
  • Unique Protocols: By running games with different random seeds, the researchers found that every pair of agents invents a completely unique language. There is no "universal AI protocol" that agents naturally converge upon.

Results and Performance

The translation quality was evaluated using standard NLP metrics and vision-language alignment scores:

  • Translation Quality: The Inter-category EC achieved a BLEU score of 9.21 and a ROUGE-L of 0.370. For comparison, a Chinese-to-English baseline on the same dataset scored 5.65 BLEU, highlighting the effectiveness of the EC translation.
  • Semantic Grounding: The translated sentences achieved a CLIP score of 0.191 with the target images. While lower than human captions (0.29), it significantly outperformed random baseline text (0.15), confirming the translations captured the visual essence of the images.
  • Hallucinations: The model occasionally "hallucinated" details to match the expected length of human sentences, sometimes adding unrelated objects to an otherwise accurate description.

Conclusion

This research demonstrates that unsupervised machine translation is a powerful tool for deciphering emergent AI communication. By revealing how environmental complexity dictates linguistic structure, this work paves the way for more interpretable multi-agent systems and potentially offers a methodology for deciphering non-human communication in other fields, such as biology.

Summary updated

Abstract

Emergent Communication (EC) provides a unique window into the language systems that emerge autonomously when agents are trained to jointly achieve shared goals. However, it is difficult to interpret EC and evaluate its relationship with natural languages (NL). This study employs unsupervised neural machine translation (UNMT) techniques to decipher ECs formed during referential games with varying task complexities, influenced by the semantic diversity of the environment. Our findings demonstrate UNMT's potential to translate EC, illustrating that task complexity characterized by semantic diversity enhances EC translatability, while higher task complexity with constrained semantic variability exhibits pragmatic EC, which, although challenging to interpret, remains suitable for translation. This research marks the first attempt, to our knowledge, to translate EC without the aid of parallel data.
BibTeX
@inproceedings{levy2025unsupervised,
  author = {Ido Levy and Orr Paradise and Boaz Carmeli and Ron Meir and Shafi Goldwasser and Yonatan Belinkov},
  title = {{Unsupervised Translation of Emergent Communication}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=GKiu4RvnCb},
}
Figure