On Temporal References in Emergent Communication

Olaf Lipinski, Adam J. Sobey, Federico Cerutti, Timothy J. Norman

Abstract

This paper investigates the conditions under which autonomous agents develop temporal references—the ability to communicate about past events—when learning a language from scratch. Through a series of experiments in a novel Temporal Referential Game, we demonstrate that while specific reward structures are insufficient, a fundamental architectural shift toward sequential data processing allows temporal language to emerge naturally in over 95% of cases. These findings suggest that the capacity for temporal displacement in language is primarily a function of an agent's internal temporal representation rather than external incentives.

Introduction

In the field of Emergent Communication (EC), researchers study how AI agents develop their own protocols to solve collaborative tasks. Historically, this research has focused on descriptive language—how agents identify and label static objects (e.g., "red circle"). However, human language is characterized by "displacement," the ability to refer to things not present in the immediate environment, including past events.

Until now, there has been a significant gap in understanding how agents might develop these temporal markers. Without the ability to signal that a target is "the same as before," agents are forced to re-describe every object from scratch, leading to communicative inefficiency. This study asks: What are the prerequisites for agents to start talking about time? Is it the environment, the reward structure, or the underlying architecture of the agent?

The Temporal Referential Game

To study this phenomenon, the researchers introduced the Temporal Referential Game (TRG), a variation of the classic Referential Game (RG).

Comparison between the standard Referential Game (a) and the Temporal Referential Game (b).

In a standard RG, a Sender describes a target object, and a Receiver must identify it from a set of distractors. In the TRG, the target object has a 50% chance of being a repeat from a previous round (within a specific temporal horizon). This setup creates a functional incentive for the agents to develop a shorthand for "the same as $N$ steps ago."

Agent Architectures

The study compares a standard baseline against two novel architectures designed to handle temporal data:

  1. Base Agent: Built on the EGG (Emergent Games Group) framework, this agent uses an LSTM network but processes data in parallel batches. While efficient for training, this method prevents the agent from maintaining a continuous hidden state across chronological steps.
  2. Temporal Agent: This model utilizes sequential batching. By processing objects one after another in a sequence, the LSTM's hidden state can carry information from one time step to the next, providing an inherent sense of history.
  3. TemporalR Agent: A hybrid model that uses parallel batching for individual object features but sequential batching to track temporal relationships.
The neural architectures for the Sender (left) and Receiver (right) agents.

In addition to architectural changes, the researchers tested a Temporal Prediction Loss. This auxiliary reward penalizes agents if they cannot correctly identify when an object last appeared, effectively "nudging" them toward temporal awareness.

Key Insights and Results

Architecture Trumps Loss

The most significant finding is that changing the reward function is insufficient for the emergence of temporal language. Agents with parallel architectures (Base) failed to develop temporal references (0% emergence) even when given specific temporal rewards. Conversely, agents with sequential architectures developed temporal references in over 95% of cases, even without the auxiliary loss. This suggests that the ability to represent sequences internally is the primary prerequisite for temporal communication.

The "Previously" Operator

Qualitative analysis of the emergent protocols revealed that agents effectively reinvented a form of Past Linear Temporal Logic (PLTL). They developed specific, short messages that functioned like the $\ominus$ ("previously") operator. These messages signaled "the target is the same as the one $N$ steps ago," allowing for highly efficient communication.

Efficiency and Generalization

Temporal references act as a form of linguistic compression. Mirroring Zipf’s Law in human language, the most frequent meanings (like "repeat") were assigned the shortest messages. Furthermore, these temporal markers improved generalization; an agent that understands "same as last time" can communicate about a completely novel object it has never seen before, provided that object is a repetition.

Discussion and Limitations

While the emergence of temporal language was highly successful, the researchers noted that in "Hard" variants—where targets and distractors are visually very similar—accuracy dropped to between 72% and 85%. Temporal references did not provide a significant performance boost in these cases, suggesting that perceptual similarity remains a major bottleneck. If the Receiver fails to identify an object correctly the first time it appears, any subsequent temporal reference to that object becomes useless.

Future work will explore more complex environments, such as "Social Deduction" games (e.g., Among Us), where referencing past statements and actions is critical for success. The authors also aim to combine these architectural insights with linguistic parsimony pressures to encourage even more efficient and human-like coding strategies.

Summary updated

Abstract

Emergent communication studies the development of language between autonomous agents, aiming to improve understanding of natural language evolution and increase communication efficiency. While temporal aspects of language have been considered in computational linguistics, there has been no research on temporal references in emergent communication. This paper addresses this gap, by exploring how agents communicate about temporal relationships. We analyse three potential influences for the emergence of temporal references: environmental, external, and architectural changes. Our experiments demonstrate that altering the loss function is insufficient for temporal references to emerge; rather, architectural changes are necessary. However, a minimal change in agent architecture, using a different batching method, allows the emergence of temporal references. This modified design is compared with the standard architecture in a temporal referential games environment, which emphasises temporal relationships. The analysis indicates that over 95\% of the agents with the modified batching method develop temporal references, without changes to their loss function. We consider temporal referencing necessary for future improvements to the agents' communication efficiency, yielding a closer to optimal coding as compared to purely compositional languages. Our readily transferable architectural insights provide the basis for their incorporation into other emergent communication settings.
BibTeX
@misc{lipinski2023temporal,
  author = {Olaf Lipinski and Adam J. Sobey and Federico Cerutti and Timothy J. Norman},
  title = {{On Temporal References in Emergent Communication}},
  year = {2023},
  eprint = {2310.06555},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.06555},
}
Figure