Echo Checks Against Two-Faced Teammates in Multi-Agent Learning

Teams of robots, drones or vehicles that each see only part of the world must share what they see to act together. That sharing opens a specific weakness. A faulty or hijacked teammate can tell different partners different things: "the target is east" to one and "west" to another. Each recipient gets a message that looks plausible on its own, so the team splits. Distributed-systems researchers call this equivocation. Their standard defense is a full agreement protocol, which is far too slow for step-by-step control on small devices. Learning-based multi-agent methods have no defense against it at all.

We propose a much cheaper middle ground. After receiving a signed message, each agent forwards a short fingerprint of it (a few-byte hash) to a few other recipients of the same sender. Two different signed fingerprints from one sender for the same time step prove that the sender lied, and that proof can be passed to any teammate without a vote. A learned gate decides, message by message, whether the policy should wait one step for this cross-check or act immediately on an unverified message. Our claim is that this cross-check matters most exactly when partial observability is worst. When teammates' views overlap little, no agent can catch a lie by comparing it with what it sees itself, and comparing notes with other recipients is the only way left.

Core Claim

Cooperative control under partial observability does not need the full machinery of consensus. It needs one piece of it: non-equivocation, meaning every recipient of a message gets the same message. It needs this only for messages that feed decisions, and only among agents whose actions interact.

A one-round digest echo keeps this safety property. It adds a constant number of short forwards per message and at most one step of delay. Full ordering protocols instead need many rounds, and their latency grows quickly with team size. We make three testable claims. First, under equivocation attacks the echo recovers most of the return lost to an attacker, at a small fraction of the latency and bandwidth of the DAG consensus of Wu et al.. Second, the echo's advantage over per-agent trust filtering grows as observation overlap shrinks. Third, the echo gives no advantage against a Byzantine agent that tells everyone the same lie. That third, negative prediction marks where the method stops helping. We expect the right design to pair the echo with a belief filter rather than replace the filter.

Where This Sits

The consensus side supplies the mechanism. Wu et al. separate the spreading of data from the ordering of messages in a two-dimensional DAG (a directed acyclic graph of who has seen which message) for wireless connected vehicles. They enforce non-equivocation before ordering and repair lost packets by querying neighbors. They report tolerating a loss proportion of 0.792 at 90% consensus success with 20 nodes. Latency, however, grows from 105 ms at 4 nodes to over 5 seconds at 32. This motivates the proposal: their non-equivocation layer is what we want, and their ordering and repair rounds are what we cannot afford. We keep the first and drop the second.

Our identity assumption follows the lightweight line. Proof-of-Authority validation on smart meters (Qing Yang et al.) replaces computational work with pre-assigned identities, and we likewise give each agent a signing key at deployment. We do not adopt a fixed committee, such as the endorser nodes of G-PBFT (Lao et al.). A fixed committee would reintroduce a single point of failure in mobile teams.

The learning side supplies the setting and the competitors. Three lines address missing or untrustworthy information, and each leaves a gap. MARO (Santos et al.) fills in dropped teammate observations with a learned predictive model but trusts every message it does receive. EIR-MAPPO (Simin Li et al.) has each agent keep a Bayesian belief about whether each teammate is an adversary. Those beliefs are private, so each agent sees only its own slice of an equivocating sender's behavior. SOG (Shao et al.) routes follower observations through elected "conductors" and assumes perfect channels. Its authors state as a limitation that a failing conductor breaks its subgroup, and a Byzantine conductor that equivocates is an obvious attack on it. ExpoComm (Xinran Li et al.) gives a linear-cost exponential-graph communication pattern that we reuse as the transport for large teams. It has no integrity checks of any kind.

Research Program

The base system is a communicating multi-agent policy trained with centralized training and decentralized execution. During training a critic sees the global state; at run time each agent acts from its own observations and received messages.

At each step, agent $i$ sends a signed message to its recipient set. The recipient set is all teammates on small maps, or the exponential-graph peers of ExpoComm on large ones. Each recipient piggybacks the sender's signed digest onto its own next outgoing message to $k$ other recipients of $i$, with $k$ between 1 and 3. A recipient then labels each incoming message as confirmed, conflicting or unconfirmed:

  • Confirmed: it collected $q$ matching echoes before a one-step deadline.
  • Conflicting: it saw two different signed digests from $i$ for the same step. It forwards both as proof, and every honest agent that verifies the proof masks $i$ for the rest of the episode.
  • Unconfirmed: anything else.

Unconfirmed and missing messages go to a MARO-style predictor, which fills in a replacement.

The gate is a small head on each agent's recurrent policy. For each incoming message it chooses between acting now on the raw content and waiting one step for the verdict. It is trained with the team return minus a per-message cost, and the cost weight is tuned by a Lagrange multiplier so that the average bytes per agent per step stay under a fixed budget. This is the "resource-constrained" part made explicit: bandwidth and delay are paid only where the policy has learned that verification changes the action.

Adversaries are trained as in EIR-MAPPO, with the attacker's policy updated faster than the team's. There are three attacker classes: an equivocator that sends different content to different recipients, a consistent liar, and a silent dropper. The number of Byzantine agents $f$ ranges from 0 to about a quarter of the team.

A digest echo cannot prevent equivocation. It can only detect it, and detection requires two recipients of conflicting messages to be linked by an echo path. We will therefore measure detection probability as a function of $k$, packet loss and $f$. With independent echo loss, the chance that a two-way split goes unnoticed should fall roughly geometrically in $k$ times the fraction of echoes delivered. This follows from treating each echo as an independent chance to link the two recipient groups, and it is the quantity we will check against simulation.

Baselines

The naive reference is Finetuned-QMIX with no communication (Hu et al.). It uses the tuned implementation that reports an average win rate of 94.9% across hard SMAC maps and cannot be attacked through messages at all. If our communicating agents under attack fall below it, communication is a liability and the proposal fails. It is run with its published tuning, on the same maps and step budget.

The strongest competitor on our specific question is EIR-MAPPO (Simin Li et al.), placed on the same communicating backbone. It reports 25.45% higher average reward than baselines under attack on SMAC 4m vs 3m and about 5.8% on Level-Based Foraging. It is the strongest competitor because it already handles untrustworthy teammates under partial observability without any protocol. It gets the same adversary training schedule, environment steps and hyperparameter search budget as our method. Beating it means higher return under equivocation attacks, with non-overlapping confidence intervals over at least 5 seeds, at equal or lower bytes per step.

The second protocol-free competitor is MARO (Santos et al.), run from the hybrid-marl code (PPSantos/hybrid-marl), which reports beating baselines in 22 of 24 dropout scenarios. We expect it to match us under pure dropout and fail under equivocation. Matching it with no attacker present is also a requirement, since the echo must not cost clean performance.

The heavy reference is the full two-dimensional DAG consensus of Wu et al. placed in the control loop. The policy acts only on consensus-committed messages and waits otherwise, under the same simulated wireless loss model. This bounds what full agreement buys. Beating it means comparable return under attack with much lower latency, particularly at 32 agents, where its reported latency exceeds 5 seconds.

Two ablations isolate the key ingredients. Signatures without echo tests whether authentication alone suffices; it should not, because a sender can sign two different messages. Echo without the learned gate, where the policy always waits, tests whether the gate buys anything under the bandwidth budget. We also report EIR-MAPPO combined with the echo, which we expect to be the best overall system.

Why not just…?

Why not just drop communication and use a well-tuned CTDE method? On standard SMAC this is a serious threat, since Finetuned-QMIX already reaches 94.9% on hard maps (Hu et al.). But the evidence in the file says it breaks when views are genuinely complementary. SOG reports that non-communicating baselines collapse when sight range drops to 0.5 (Shao et al.), and local-only agents fail on HearSee in Santos et al.. We therefore report results on both default SMAC and reduced-sight-range variants, and we concede the default setting if no communicating method beats it there.

Why not just let each agent filter untrustworthy teammates with Bayesian beliefs, as EIR-MAPPO does? This is the real threat. It will catch lies that contradict what the recipient sees itself, and it needs no protocol. It cannot catch a split message when the two recipients see disjoint parts of the map, because each half of the lie is consistent with its recipient's view. The observation-overlap sweep in our experiments settles this directly. If EIR-MAPPO matches the echo at low overlap, our central claim is wrong.

Why not just run full DAG or PBFT consensus on the messages? It gives the strongest guarantees, and Wu et al. show impressive loss tolerance. Their own numbers, 105 ms at 4 nodes rising past 5 seconds at 32, are incompatible with control loops that act many times per second. Our latency-versus-team-size curve will show whether the extra guarantees of ordering buy any return beyond non-equivocation.

Why not just take a median or vote over received messages? Voting needs redundancy of the same content. Under partial observability, different senders report different things, so a median across them has no meaning. Only copies of the same sender's message can be compared, and that is precisely the echo.

Proposed Experiments

We use three testbeds. The first is the hybrid-marl MPE and Level-Based Foraging tasks, including HearSee, SpreadXY and SpreadBlindfold. They have complementary views by construction and support per-link dropout. The second is SMAC (Samvelyan et al.) with a communication channel added, sight range swept from default down to half, and the 4m vs 3m attack setting of EIR-MAPPO for direct comparison. The third is MAgent Battle and Pursuit (Zheng et al.) over ExpoComm's one-peer graph with 20 to 100 agents; the file reports no score on these tasks for ExpoComm, so we will run it ourselves as the transport baseline. Across all testbeds a simulated wireless layer applies per-packet loss from 0 up to 0.8, matching the range in Wu et al., together with per-hop delay.

We report five kinds of measurement:

  • Performance: win rate or return under each attacker class.
  • Detection: detection rate and time to detect an equivocator.
  • Resources: bytes and messages per agent per step, decision latency in simulated milliseconds, and per-agent memory for keys and digests.
  • Cost of caution: the fraction of messages the gate chooses to verify.

Three checks decide the proposal. The overlap sweep must show the echo's advantage over EIR-MAPPO widening as sight range shrinks under equivocation, and vanishing under the consistent-liar attack. The latency curve must stay nearly flat in team size while the in-loop DAG baseline grows. The gate must reach at least the always-wait variant's attacked return within the bandwidth budget, while verifying only a minority of messages. If the first check fails, the idea reduces to a cost-saving variant of belief filtering, and we would say so.