When neural agents learn to talk by passing real-valued vectors and backpropagating through each other, they coordinate quickly. The cost is that nobody can read what they say. This proposal tests a specific way to keep the speed and recover readability. Each message is built mostly as a soft pointer to items in a knowledge graph: an entity, a relation, and another entity, which together form one fact. A small free-form vector is added to that pointer. The receiving agent does not treat the message as an opaque vector. It places the message inside its own graph at the entity the message points to, and its attention is biased by graph distance from that spot. Readable content is then structurally cheaper to use than the free-form part. The bet is that gradient descent will route information through the readable part without a heavy penalty forcing it there. Afterwards, every message can be decoded as a short list of triples and checked against the sender's graph.
The setting makes communication genuinely necessary. Two or more agents each hold a different slice of a knowledge graph and must jointly answer queries whose evidence path crosses slices. Partitioned graphs arise from privacy constraints, organizational boundaries, or incompleteness. In this setting a single centralized model is either not allowed or not available. A model that reads one fact in isolation also cannot resolve the ambiguity that only the other agent's neighborhood would settle. The claim is narrow and testable. At matched task accuracy, triple-anchored channels should produce messages whose decoded triples are true facts lying on the gold evidence path. Their accuracy should depend little on the free-form part. Free continuous channels, quantized channels read after the fact, and text exchange should each fall short on at least one of these measures.
The Gap This Fills
Differentiable communication, meaning agents linked inside one training graph so the receiver's loss gradient flows into the sender, has a long track record. It began with DIAL in Learning to Communicate with Deep Multi-Agent Reinforcement Learning and broadcast channels in Learning Multiagent Communication with Backpropagation. It continued with learned addressing in TarMAC: Targeted Multi-Agent Communication and, most recently, with language models joined by trainable attention edges in Language Model Networks. That last paper reports a +30.5% relative gain over prompting and states plainly that the resulting intermediate representations are unreadable. All of these channels are free vectors, so the content has no reference point.
What Do Latent Agents Actually Represent? supplies the motivating evidence that free vectors are not just opaque but often idle. In a four-agent latent system, 93.6% of the 2,560 channel dimensions were inactive, and cross-agent redundancy reached an R² of 0.75 to 0.80. The authors read the structure as the base model's static geometry rather than collaboration. This suggests that most of a continuous channel's capacity is unused. A channel with far fewer free degrees of freedom, anchored to explicit referents, may therefore not cost much accuracy. That is the hypothesis we test.
The emergent-communication literature offers two partial remedies, and both constrain our design. Emergent Quantized Communication snaps messages to discrete bins with a straight-through gradient. It keeps above 92% accuracy with up to 10,000 distractors, whereas Gumbel-Softmax collapsed to 58 unique messages for 1,000 targets. That result rules out stochastic discrete sampling as our discretization route. Concept-Best-Matching maps emergent tokens to ground-truth concepts by bipartite matching and finds up to 0.55 of concepts unmatched under Gumbel-Softmax on THING. That result shows that meaning recovered after training is fragile even for flat attribute concepts. Both methods give discrete symbols whose meaning must be discovered after training. Our channel's symbols are knowledge-graph items by construction. What remains to verify is whether the agents use them truthfully, not what they mean.
On the graph side, Towards Graph Representation Learning in Emergent Communication gave signalling agents graph-convolutional encoders over synthetic star graphs. It showed compositional vocabulary use, but with single-symbol messages and no relational knowledge. Emergent Communication with Attention showed that speaker and listener attention acts as a bottleneck that improves generalization and compositionality, but over image patches with no structural constraint.
Two further results motivate the receiver design. SCoP and CoLAKE show that reading a triple together with its neighborhood beats reading it alone; CoLAKE does this with hard adjacency masks inside the Transformer. SMiLE and Orthogonal Relation Transforms with Graph Context Modeling show that isolated triples cannot resolve many-to-many relations or multi-faceted entities. Graphormer supplies the mechanism we reuse: a learned bias, indexed by shortest-path distance, added to attention scores. None of these works places another agent's message inside the receiver's graph. The scene-graph semantic communication system in Scene Graph-Aided Probabilistic Semantic Communication does transmit triples, but its protocol is hand-engineered rather than learned.
Core Claim
Free continuous messages are opaque because they have no anchor in a structure the receiver also has. Suppose instead that messages are soft pointers into a shared entity and relation vocabulary, and that the receiver interprets them through attention constrained by its own graph's topology. Then gradient-trained agents will carry nearly all task-relevant content in the pointer rather than in the free part. Their decoded messages will be the cross-partition facts and short paths the receiver lacks. The sender, in turn, uses topology-constrained attention over its own neighborhood to pick which facet of an entity to send. This is exactly the context the isolated-triple literature says is needed.
Mechanism
Each agent encodes its partition subgraph, conditioned on the query, with a Graphormer-style Transformer using shortest-path biases. The sender emits $K$ message slots. Slot $k$ is
$$m_k = \sum_{e} \alpha_k(e)\, E_e \;\oplus\; \sum_{r} \beta_k(r)\, R_r \;\oplus\; \sum_{e'} \gamma_k(e')\, E_{e'} \;+\; \rho_k .$$
Here $\alpha_k, \beta_k, \gamma_k$ are softmax distributions over shared entities, relations, and tail entities, produced by the sender's attention over its own nodes. $E$ and $R$ are shared embedding tables. $\oplus$ denotes concatenation. $\rho_k$ is a low-dimensional residual whose norm is mildly penalized. Entity vocabularies overlap across partitions, as in entity alignment, so pointers have referents on both sides.
The receiver adds slot $k$ as a virtual token attached to its own copy of the entity the slot points to. It attends from every node $j$ to that token with score $q_j^\top \kappa_k/\sqrt{d} + \sum_e \alpha_k(e)\, b_{\phi(j,e)}$. Here $q_j$ is node $j$'s query, $\kappa_k$ is the slot's key, and $b_{\phi(j,e)}$ is the learned Graphormer bias for the shortest-path distance between $j$ and entity $e$ in the receiver's graph. Messages about nearby entities thus enter the receiver's reasoning locally. Everything stays differentiable, so training is end-to-end in the DIAL sense.
Research Program
The work has three parts.
The first is building the partitioned benchmarks. We start with a synthetic knowledge-graph generator with planted multi-hop rules whose evidence paths are forced to cross partitions, so gold message content is known exactly. We then partition real graphs: the incomplete-graph question-answering benchmark IKGWQ from Debate over Mixed-knowledge, and WN-all entity type prediction from SCoP. We split relations or entity sets across two to four agents.
The second is training the anchored channel alongside every baseline under a shared compute and tuning budget.
The third is measuring readability directly. We report three quantities:
- Triple precision: the fraction of argmax-decoded triples that are true facts in the sender's partition.
- Path recall: the fraction of gold cross-partition evidence edges that appear among decoded messages.
- Residual dependence: the accuracy drop when $\rho$ is zeroed at test time.
We also measure pointer-shape leakage: the accuracy drop when the soft pointers are replaced by their hard argmax. This detects agents hiding information in the low-probability tail of the distributions, a form of steganography we expect to be the main failure mode.
Baselines
The naive reference point is no communication. Each agent answers from its own partition, and accuracy is taken from the best single agent. Any channel must beat this clearly on cross-partition queries, or the benchmark does not require communication. The centralized upper bound is a Graphormer-style model, plus SCoP on WN-all, trained on the union of partitions. SCoP reports 91.98% Hit@1 on WN-all entity type prediction, though in the unpartitioned setting. We do not expect to beat the upper bound; we report the fraction of its accuracy each channel recovers.
The strongest accuracy competitor is a free continuous channel. It uses TarMAC-style targeted attention between agents, the same encoders, and message size equal to the anchored channel's total width. If the dead-dimension finding of Lu et al. transfers, this baseline should match but not greatly exceed anchored accuracy.
The strongest readability competitor is the quantized channel of Emergent Quantized Communication, read afterwards with Concept-Best-Matching against the knowledge graph's triples. It gets identical encoders, budget, and an equal number of message slots.
The strongest text-based competitor is DoM-style debate between language-model agents exchanging verbalized triples. DoM reports 84.5% Hits@1 on IKGWQ, against 49.5% for GoG, in an unpartitioned setting with large language models. We will both quote that number and rerun DoM on our partitions with a matched model size. On WN-all, there is no reported result found on Lacuna for any multi-agent or partitioned setting.
Three ablations isolate the key ingredient. In the first, receiver placement is flat: pointers enter as plain tokens with no shortest-path bias. In the second, the residual is removed. In the third, the residual is unpenalized. The anchored channel counts as winning on two conditions. It must stay within a few points of the free channel's accuracy. And it must beat the quantized baseline on triple precision and path recall by a margin larger than seed variance, with residual dependence and pointer-shape leakage both small.
Why not just…?
Why not just quantize the channel and map tokens to triples afterwards? This is the most serious alternative. Emergent Quantized Communication already matches continuous accuracy with discrete symbols. Concept-Best-Matching could then align those symbols to knowledge-graph items. Against it, CBM's own results show large unmatched fractions even for 50 flat concepts. A knowledge graph has thousands of entities and dozens of relations, and one integer token would have to stand for a composite fact. We will settle this by running both at matched accuracy and comparing triple precision and path recall. If quantization plus matching reaches comparable readability, the anchored channel's contribution reduces to convenience, and we will say so.
Why not just have agents send verbalized triples as text? Text is readable by construction, and DoM shows debate is strong on incomplete-graph question answering. However, hard symbol selection breaks gradient flow. Language Model Networks reports that differentiable communication beat text-based test-time methods such as Self-Consistency and Self-Refine at comparable budget. The "hard argmax pointer" evaluation of our own channel is effectively a learned triple-passing protocol. If DoM on our partitions matches anchored accuracy at similar inference cost, the differentiable component is not worth it on this task.
Why not just centralize the graph? Where pooling is allowed, one should. The centralized model is our upper bound, not a competitor. The proposal is relevant only where partitions cannot be merged, and its value is measured as the fraction of centralized accuracy recovered.
Why not just probe a free continuous channel afterwards? Linear probes or belief matching in the style of Andreas et al. ( Translating Neuralese) can find triple information in free messages. Finding information is weaker than the message being that information, because nothing stops the receiver from using unprobed directions. We will probe the free channel for gold triples and report how much accuracy survives when messages are projected onto the probed subspace. The comparison is against the residual-ablation drop for anchored messages.
Decisive Checks
On the synthetic benchmark with planted cross-partition rules, we vary the number of hops crossing partitions from one to three. The anchored channel should show path recall rising with correct answers, and it should show near-zero residual dependence. If the residual or pointer tails carry most of the signal, the core claim fails.
The flat-placement ablation tests the mechanism. If removing shortest-path placement at the receiver leaves readability unchanged, then the topology constraint is not what pushes content into the pointer, and the anchoring alone is doing the work.
On IKGWQ partitions, the anchored channel should approach the free channel's Hits@1 and beat the quantized-plus-matching baseline on triple precision. We report DoM's 84.5% as context rather than a like-for-like target.
Finally, we test the isolated-triple claim on many-to-many relations and multi-faceted entities. Decoded sender messages should pick the facet relevant to the query, for example filmography rather than nationality. Message relation distributions $\beta_k$ should shift with query relation even for the same anchored entity. If they do not, the sender is transmitting context-blind facts, and the benefit of topology-aware sending is absent.
References
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Learning Multiagent Communication with Backpropagation
- TarMAC: Targeted Multi-Agent Communication
- Language Model Networks: Supervision-Efficient Learning through Dense Communication
- What Do Latent Agents Actually Represent? Interpreting Hidden-State Communication in Multi-Agent Systems
- Emergent Quantized Communication
- Concept-Best-Matching: Evaluating Compositionality In Emergent Communication
- Towards Graph Representation Learning in Emergent Communication
- Emergent Communication with Attention
- Improving Knowledge Graph Representation Learning by Structure Contextual Pre-training
- CoLAKE: Contextualized Language and Knowledge Embedding
- SMiLE: Schema-augmented Multi-level Contrastive Learning for Knowledge Graph Link Prediction
- Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding
- Do Transformers Really Perform Badly for Graph Representation?
- Scene Graph-Aided Probabilistic Semantic Communication for Image Transmission
- Debate over Mixed-knowledge: A Robust Multi-Agent Reasoning Framework for Incomplete Knowledge Graph Question Answering
- Translating Neuralese
- Translating Neuralese