TL;DR
AI agents can naturally learn to communicate about the past, but only if their underlying neural architecture is explicitly wired to process time sequentially. By introducing a simple shift from parallel to sequential batching in recurrent networks, this paper demonstrates that temporal references (e.g., "the object from two turns ago") reliably emerge in multi-agent languages without needing explicit changes to the loss function.
What Problem Does This Paper Address?
In emergent communication research, autonomous AI agents are placed in environments where they must invent a language from scratch to coordinate and solve tasks. While researchers have successfully simulated the emergence of vocabulary alignment and compositional grammar (like combining "blue" and "circle"), these languages have remained strictly stuck in the present.
In contrast, human language relies heavily on temporal structure—we constantly use tense and time-based references ("as I mentioned before", "two days ago") to compress information and boost bandwidth efficiency. Prior to this paper, there was no documented evidence of temporal references emerging spontaneously in multi-agent communication. The authors set out to find the exact ingredients—environmental pressures, external loss functions, or architectural biases—required to make AI agents understand and talk about time.
Methods
The authors modified the classic "referential game" framework to create the Temporal Referential Game (TRG). In a standard referential game, a sender agent looks at a target object and sends a message to a receiver agent, who must pick out the target from a lineup of random distractors. In the TRG variant, the target object has a specific probability of being a repeat of an object seen $n$ steps ago in the past.
To test what drives temporal language, they evaluated three agent architectures:
- Base Agent: The standard framework where an LSTM processes each game episode independently in parallel (input batch shape:
[batch_size, 1, attributes]). - Temporal Agent: A novel design featuring a sequentially batched LSTM that processes objects in chronological order (input batch shape:
[1, batch_size, attributes]), allowing it to maintain a continuous historical state. - TemporalR Agent: A hybrid architecture combining the parallel understanding of the Base agent with the sequential module of the Temporal agent via element-wise multiplication.
They also introduced a strict Temporality Metric ($M_{\ominus n}$) to track whether a specific message is used exclusively as a temporal operator (acting like the word "previously") rather than just a static description of an object's features.
Datasets
The agents were trained and evaluated across six distinct environment variants designed to isolate temporal patterns:
- TRG / TRG Hard: The core temporal referential games where targets repeat from a previous horizon ($h \le 8$). In the "Hard" variant, distractors differ from the target by only a single attribute.
- RG / RG Hard: Classic referential games with zero intentional target repetitions, serving as baselines.
- Always Same: A verification environment where a subset of targets is repeated sequentially to test for consistent temporal messaging.
- Never Same: A sanity-check environment with absolutely no target repetitions to ensure the temporality metric doesn't flag false positives.
Key Insights and Ideas
- Architecture Trumps Loss: The most profound takeaway is that tweaking the reward system (adding an auxiliary loss to predict how many steps ago an object was seen) is neither sufficient nor necessary for temporal language to emerge. Instead, architectural bias is the true catalyst. Agents only talk about time if their neural network batching mechanism physically allows them to step through time sequentially.
- Natural Efficiency (Zipf's Law): When given the architectural capacity, agents naturally realize that sending a single specialized message meaning "the object from last turn" takes less bandwidth than describing all the individual attributes of that object from scratch.
- Zero Trade-offs with Compositionality: Introducing temporal references did not degrade the language's compositionality (measured via topographic similarity). The agents successfully integrated abstract temporal markers without breaking their ability to describe static features.
Key Results
- 100% Success Rate: 100% of the Temporal agents and over 96% of the TemporalR agents successfully developed perfect temporal references ($M_{\ominus n} = 100\%$) across all training environments.
- Standard Agents are Time-Blind: The standard Base agents achieved 0% temporal reference emergence in every single test case, proving that standard parallel-batching paradigms completely blind agents to historical context.
- The Accuracy Paradox: Surprisingly, developing temporal references did not improve the agents' final task accuracy in the games. Both the base and temporal agents performed similarly (~72% on RG Hard, ~85% on TRG Hard). The authors attribute this to "perceptual similarity"—if the receiver agent fails to correctly identify an object the first time it appears, a temporal reference to that past failure cannot save it.
Limitations and Future Work
A key limitation noted by the authors is that standard compositionality metrics (like posdis and bosdis) rely on direct mappings between messages and static dataset attributes. Because temporal messages map to time steps rather than object properties, they are falsely penalized as "non-compositional" by current evaluation benchmarks.
For future work, the authors suggest moving beyond simplified referential games into complex, strategically rich environments—such as multi-agent social deduction games—to explore how temporal communication handles richer dynamics, deception, and long-term planning.
Summary updated