BE-CTDE: Blockchain-Enabled Multi-Agent Reinforcement Learning with Anomaly Detection and Token Economics

Chih-Wei Huang, Hsin-Chieh Lee, Yun-Chen Yu, Ibrahim Althamary, Rongxing Lu, Phone LinarXiv 2026 · arXiv

TL;DR

BE-CTDE introduces a blockchain-based management layer for multi-agent reinforcement learning (MARL) that uses behavioral verification and token economics to mitigate adversarial disruptions. By linking an agent’s token balance to its influence on global policy updates, the framework ensures that malicious behavior is not only detected but also economically penalized and systematically excluded from the learning process.

What Problem Does This Paper Address?

In the Internet of Things (IoT), multi-agent systems like drone swarms or autonomous traffic grids rely on cooperative learning to function. The standard framework for this is Centralized Training and Decentralized Execution (CTDE). However, CTDE has a "trust gap": the centralized trainer is a single point of failure, and individual agents can "disobey" the learned policy—either due to hardware failure or malicious intent (e.g., data poisoning or behavioral manipulation).

While previous research has suggested using blockchain to audit these systems, most existing solutions are "open-loop"—they record what happened but don't use that information to protect the training process in real-time. An agent caught acting maliciously might still have its "poisoned" data included in the next global model update.

Methods

The authors propose BE-CTDE, a closed-loop system where the blockchain acts as an active training manager.

Fig. 1. BE-CTDE system architecture: IoT agents interact with the environ- ment through observations ot, rewards rt and actions at, while the blockchain layer coordinates training, verifies behavior and manages token incentives. Neural-network computation remains off-chain.

Key components include:

  • Behavioral Consistency Verification: The system compares an agent's actual executed trajectory against what the shared policy recommended it should do. It calculates a "behavioral residual" to identify deviations.
  • Token Economics: Agents must stake tokens to participate. If the behavioral verifier flags an agent, its stake is slashed. A portion is burned, and a portion is redistributed as "compensation" to honest agents.
  • Token-Weighted Aggregation: This is the core innovation. When updating the global policy, the contribution (gradient) of each agent is weighted by its current token balance.
  • Economic Exclusion: If an agent's balance falls below a certain threshold, it is permanently excluded from contributing to the global model.

Crucially, the heavy lifting of neural network training (PPO, advantage estimation) happens off-chain to remain feasible for IoT devices, while the coordination and accounting happen on-chain via Ethereum smart contracts.

Fig. 2. BE-CTDE training workflow and shared-policy architecture. Neural inference, communication, advantage estimation and PPO run off-chain (or- ange), while behavioral verification and token settlement run on-chain (blue); document icons mark results committed to the chain.

Datasets

The framework was evaluated using a Multi-Agent Path Finding (MAPF) simulation environment built on the state-of-the-art SCRIMP backbone.

  • Environment: A 40x40 grid with varying obstacle densities (0% to 30%).
  • Population Scales: Tested with 8, 32, 64, and 128 agents.
  • Adversarial Models: Malicious agents were programmed for static blocking, occupying goal regions, or repetitive/stagnant movement to disrupt the group.

Key Insights and Ideas

  • Consequential Auditability: The paper argues that auditability is useless without consequence. By feeding economic state back into the gradient aggregation step, the system automatically "mutes" the influence of suspicious agents before they can be fully excluded.
  • The "Capacity Bound": The authors derive a mathematical limit on how many agents a population-relative detector can flag at once. Because malicious agents shift the group's average behavior, they can "mask" themselves. This insight explains why multi-round exclusion is necessary.
  • Imperfect Detection is Sufficient: A major theoretical contribution is proving that you don't need a perfect detector to secure the system. As long as the detection rate stays above a specific threshold (the "detection floor"), adversaries will eventually be excluded in finite time.

Key Results

The researchers compared BE-CTDE against the undefended SCRIMP model and several partially defended baselines.

  • Success Rates (SR): Under attack with 8 agents, BE-CTDE maintained a 100% success rate, while the undefended SCRIMP baseline dropped to 0%.
  • Resilience to Majority Attacks: Even when 62.5% of the agents were malicious, BE-CTDE achieved success rates of at least 93% for populations up to 32 agents.
  • Collision Reduction: BE-CTDE reduced collision rates by up to 35% compared to detection without economic consequences, and by 43% compared to conventional stake slashing without weighted aggregation.

Limitations and Future Work

  • Trust Assumptions: The model currently assumes an honest training coordinator and that the physical sensor data reported to the chain is authentic. It does not yet account for sophisticated sensor spoofing.
  • Threat Model Scope: The current version does not address Sybil attacks (where one attacker creates many identities), gradient-inversion attacks, or collusion between malicious agents.
  • Latency: While off-chain execution helps, the blockchain layer introduces transaction overhead. The authors note that deployment suitability depends on the specific timing requirements of the IoT application.
  • Future Directions: The authors plan to extend the framework to broader threat models and more diverse task domains beyond pathfinding.

Summary updated

Abstract

Multi-agent reinforcement learning supports cooperative decision-making in Internet of Things systems, but strategic deviations can disrupt coordination and undermine confidence in the training process. This paper proposes Blockchain-Enabled Centralized Training and Decentralized Execution (BE-CTDE), which uses blockchain as an active training-management layer. A behavioral verifier compares executed trajectories with shared-policy recommendations; its flags drive a token mechanism combining slashing, burning, rewards, and compensation. Retained balances determine each agent's contribution to the global policy update, connecting auditable behavior to learning influence. We derive a one-round incentive condition that accounts for action-dependent compensation and the payoff from deviation, and a finite expected exclusion-time bound under explicit conditional detection and accounting assumptions, showing perfect detection is unnecessary. An exact finite-population capacity bound characterizes the limits of the population-relative detector. In multi-agent path finding experiments, BE-CTDE outperforms a state-of-the-art learning-based path planner with substantially higher success rates under attack, even when adversaries form the majority. It also outperforms partially defended baselines in all configurations, with collision rates up to 35% lower than under detection without economic consequences and up to 43% lower than under conventional stake slashing. Together, these results characterize how behavioral verification and token feedback can support accountable cooperative learning.
BibTeX
@misc{huang2026be,
  author = {Chih-Wei Huang and Hsin-Chieh Lee and Yun-Chen Yu and Ibrahim Althamary and Rongxing Lu and Phone Lin},
  title = {{BE-CTDE: Blockchain-Enabled Multi-Agent Reinforcement Learning with Anomaly Detection and Token Economics}},
  year = {2026},
  eprint = {2610.34121},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34121},
}
Figure