TL;DR
The paper introduces PACE (Procedural Abstractions for Communicating Efficiently), a neuro-symbolic framework that allows artificial agents to organically invent and learn procedural abstractions during collaborative tasks. By balancing the trade-off between message brevity and the cognitive cost of learning new terms, PACE replicates human-like language evolution, naturally stabilizing near an optimal vocabulary size.
What Problem Does This Paper Address?
When humans collaborate, our language evolves to become more efficient. If we are building a house together, we quickly stop saying "place a brick, then place another brick on top, then another" and instead just say "build a column." In AI, enabling agents to dynamically introduce these kinds of procedural abstractions is a long-standing challenge.
Previous computational models have attempted to simulate this, but they usually rely on unrealistic shortcuts—such as assuming the listening agent automatically understands a new abstraction the moment it's introduced, or pre-defining what words mean. This paper addresses this gap by forcing both the speaker (the architect) and the listener (the builder) to interactively and neurally learn the meaning of new abstract terms from scratch under the pressure of efficient communication.
Methods
PACE uses a multi-agent, neuro-symbolic approach to play a repeated "Architect-Builder" game. The interaction operates in a three-phase loop:
- Selection: The architect looks at a goal-scene and chooses a symbolic program (a sequence of instructions) to describe it. Because new abstractions are shorter but unfamiliar to the builder, the architect faces an exploration vs. exploitation dilemma. PACE models this as a contextual multi-armed bandit problem, using an $\epsilon$-greedy strategy to decide when to try new, shorter programs versus reliable, verbose ones.
- Communication: The chosen program is translated into discrete messages using Emergent Communication (EC). The architect and builder play a signaling game where policies are trained end-to-end using the Gumbel-Softmax relaxation to minimize reconstruction errors.
- Abstraction: After several rounds, the architect uses library learning (based on Minimum Description Length) to scan successful programs, identify frequently repeated sub-sequences, and compress them into a single new symbolic token (e.g., combining horizontal and vertical block instructions into a single "L-shape" token).
Datasets
The authors extended the block-building dataset from McCarthy et al. (2021):
- Grid Environment: A $9 \times 9$ grid where blocks are placed.
- Shapes: Increased from 3 to 31 unique shapes resembling uppercase and lowercase English letters.
- Scenes: 961 total goal-scenes (each containing two adjacent shapes).
- Splits: 930 training scenes and 31 test scenes (ensuring that specific pairs of shapes in the test set were never seen together during training).
Key Insights and Ideas
- Pessimism is Best: When introducing a new abstraction, initializing its reinforcement learning value (Q-value) pessimistically ($q_{init} = 0$) outperforms an optimistic initialization ($q_{init} = 1$). Optimistic agents force unlearned terms too quickly, causing communication to fail. Pessimistic agents introduce terms gradually, giving the builder time to learn.
- Natural Convergence to the Pareto Frontier: PACE does not have a hardcoded limit on vocabulary size. Instead, it naturally stops introducing new abstractions. As the language grows, the marginal benefit of adding a new term decreases, while the difficulty of learning it increases. The language naturally stabilizes near the Pareto frontier of the complexity-vs-lexicon-size trade-off.
- Survival of the Fittest (Shapes): Not all proposed abstractions survive. High-frequency sub-shapes (like a simple "tower") are easily mastered by the builder and retained. Rare, overly complex sub-shapes (like a rotated "H") are judged as too difficult to learn within the trial period and are discarded by the architect's bandit mechanism.
Key Results
The authors compared PACE against two baselines: No Abstractions (stuck with primitive actions) and Greedy (always forces the shortest possible program).
- Brevity: PACE successfully halved the average morphosyntactic complexity (program length), dropping it from an initial 10 tokens down to approximately 4.92 tokens.
- Stability: While the Greedy baseline achieved even shorter program lengths, it suffered from catastrophic drops in accuracy because it forced new terms before the builder could understand them, eventually failing to recover. PACE maintained near-perfect test accuracy throughout training due to its measured, bandit-controlled exploration.
Limitations and Future Work
While PACE successfully bridges symbolic library learning and neural emergent communication, it currently operates in a structured, 2D grid block-world with a synthetic language.
The authors note that future work should focus on:
- Scaling this approach to more complex, varied collaborative domains.
- Deepening comparisons with actual human conversational data.
- Exploring how Large Language Models (LLMs) handle these dynamic, multi-round conversational abstractions when collaborating with humans or other agents.
Summary updated