PACE: Procedural Abstractions for Communicating Efficiently

Jonathan D. Thomas, Andrea Silvi, Devdatt Dubhashi, Moa Johansson

TL;DR

The paper introduces PACE (Procedural Abstractions for Communicating Efficiently), a neuro-symbolic framework that allows artificial agents to organically invent and learn procedural abstractions during collaborative tasks. By balancing the trade-off between message brevity and the cognitive cost of learning new terms, PACE replicates human-like language evolution, naturally stabilizing near an optimal vocabulary size.

What Problem Does This Paper Address?

When humans collaborate, our language evolves to become more efficient. If we are building a house together, we quickly stop saying "place a brick, then place another brick on top, then another" and instead just say "build a column." In AI, enabling agents to dynamically introduce these kinds of procedural abstractions is a long-standing challenge.

Previous computational models have attempted to simulate this, but they usually rely on unrealistic shortcuts—such as assuming the listening agent automatically understands a new abstraction the moment it's introduced, or pre-defining what words mean. This paper addresses this gap by forcing both the speaker (the architect) and the listener (the builder) to interactively and neurally learn the meaning of new abstract terms from scratch under the pressure of efficient communication.

Methods

PACE uses a multi-agent, neuro-symbolic approach to play a repeated "Architect-Builder" game. The interaction operates in a three-phase loop:

Figure 2: Interaction between the architect and builder in PACE proceeds as follows: (1) given a goal-scene the archi- tect chooses the program to communicate, and (2) the archi- tect and builder communicate via EC. After multiple interac- tions, an abstraction is introduced. Then the loop repeats.
  1. Selection: The architect looks at a goal-scene and chooses a symbolic program (a sequence of instructions) to describe it. Because new abstractions are shorter but unfamiliar to the builder, the architect faces an exploration vs. exploitation dilemma. PACE models this as a contextual multi-armed bandit problem, using an $\epsilon$-greedy strategy to decide when to try new, shorter programs versus reliable, verbose ones.
  2. Communication: The chosen program is translated into discrete messages using Emergent Communication (EC). The architect and builder play a signaling game where policies are trained end-to-end using the Gumbel-Softmax relaxation to minimize reconstruction errors.
  3. Abstraction: After several rounds, the architect uses library learning (based on Minimum Description Length) to scan successful programs, identify frequently repeated sub-sequences, and compress them into a single new symbolic token (e.g., combining horizontal and vertical block instructions into a single "L-shape" token).

Datasets

The authors extended the block-building dataset from McCarthy et al. (2021):

  • Grid Environment: A $9 \times 9$ grid where blocks are placed.
  • Shapes: Increased from 3 to 31 unique shapes resembling uppercase and lowercase English letters.
  • Scenes: 961 total goal-scenes (each containing two adjacent shapes).
  • Splits: 930 training scenes and 31 test scenes (ensuring that specific pairs of shapes in the test set were never seen together during training).

Key Insights and Ideas

  • Pessimism is Best: When introducing a new abstraction, initializing its reinforcement learning value (Q-value) pessimistically ($q_{init} = 0$) outperforms an optimistic initialization ($q_{init} = 1$). Optimistic agents force unlearned terms too quickly, causing communication to fail. Pessimistic agents introduce terms gradually, giving the builder time to learn.
  • Natural Convergence to the Pareto Frontier: PACE does not have a hardcoded limit on vocabulary size. Instead, it naturally stops introducing new abstractions. As the language grows, the marginal benefit of adding a new term decreases, while the difficulty of learning it increases. The language naturally stabilizes near the Pareto frontier of the complexity-vs-lexicon-size trade-off.
  • Survival of the Fittest (Shapes): Not all proposed abstractions survive. High-frequency sub-shapes (like a simple "tower") are easily mastered by the builder and retained. Rare, overly complex sub-shapes (like a rotated "H") are judged as too difficult to learn within the trial period and are discarded by the architect's bandit mechanism.

Key Results

The authors compared PACE against two baselines: No Abstractions (stuck with primitive actions) and Greedy (always forces the shortest possible program).

Figure 3: Comparison between PACE, Greedy and No abstractions in terms of program length and test reward over time. Line indicates mean value and shaded regions indicate the 95% confidence interval.
  • Brevity: PACE successfully halved the average morphosyntactic complexity (program length), dropping it from an initial 10 tokens down to approximately 4.92 tokens.
  • Stability: While the Greedy baseline achieved even shorter program lengths, it suffered from catastrophic drops in accuracy because it forced new terms before the builder could understand them, eventually failing to recover. PACE maintained near-perfect test accuracy throughout training due to its measured, bandit-controlled exploration.

Limitations and Future Work

While PACE successfully bridges symbolic library learning and neural emergent communication, it currently operates in a structured, 2D grid block-world with a synthetic language.

The authors note that future work should focus on:

  1. Scaling this approach to more complex, varied collaborative domains.
  2. Deepening comparisons with actual human conversational data.
  3. Exploring how Large Language Models (LLMs) handle these dynamic, multi-round conversational abstractions when collaborating with humans or other agents.

Summary updated

Abstract

A central but unresolved aspect of problem-solving in AI is the capability to introduce and use abstractions, something humans excel at. Work in cognitive science has demonstrated that humans tend towards higher levels of abstraction when engaged in collaborative task-oriented communication, enabling gradually shorter and more information-efficient utterances. Several computational methods have attempted to replicate this phenomenon, but all make unrealistic simplifying assumptions about how abstractions are introduced and learned. Our method, Procedural Abstractions for Communicating Efficiently (PACE), overcomes these limitations through a neuro-symbolic approach. On the symbolic side, we draw on work from library learning for proposing abstractions. We combine this with neural methods for communication and reinforcement learning, via a novel use of bandit algorithms for controlling the exploration and exploitation trade-off in introducing new abstractions. PACE exhibits similar tendencies to humans on a collaborative construction task from the cognitive science literature, where one agent (the architect) instructs the other (the builder) to reconstruct a scene of block-buildings. PACE results in the emergence of an efficient language as a by-product of collaborative communication. Beyond providing mechanistic insights into human communication, our work serves as a first step to providing conversational agents with the ability for human-like communicative abstractions.
BibTeX
@inproceedings{thomas2024pace,
  author = {Jonathan D. Thomas and Andrea Silvi and Devdatt Dubhashi and Moa Johansson},
  title = {{PACE: Procedural Abstractions for Communicating Efficiently}},
  booktitle = {LanGame @ NeurIPS},
  year = {2024},
  url = {https://openreview.net/forum?id=cfVwVEn98d},
}
Figure