FAuNO: Semi-Asynchronous Federated Reinforcement Learning Framework for Task Offloading in Edge Systems

Frederico Metelo, Alexandre Oliveira, Stevo Rackovic, Pedro Ákos Costa, Claudia SoaresSubmission to ICLR 2026 · Rejected

TL;DR

FAuNO (Federated Asynchronous Network Orchestrator) is a semi-asynchronous federated reinforcement learning framework designed for decentralized task offloading across heterogeneous edge computing systems. By federating only the critic via a buffered, non-blocking aggregation mechanism while keeping actor policies local, FAuNO enables continuous training despite stragglers and significantly reduces task drops and latency compared to standard federated and heuristic baselines.

What Problem Does This Paper Address?

In edge and fog computing, resource-constrained devices (like IoT sensors or single-board computers) need to offload tasks to nearby peers or servers to minimize latency and meet strict deadlines. Fully centralized orchestration creates single points of failure and network bottlenecks. While multi-agent reinforcement learning (MARL) provides a path to decentralized decision-making, deploying it across real-world edge devices introduces major hurdles:

  1. Heterogeneity and Stragglers: Synchronous federated learning (FL) algorithms stall when slow devices (stragglers) delay global aggregation rounds.
  2. Partial Observability and Non-IID Workloads: Edge nodes only observe their local neighborhood, and different regions experience wildly different traffic patterns, causing purely independent agents to diverge into selfish, non-cooperative behaviors.

Methods

FAuNO formulates decentralized task offloading as a Partially Observable Markov Game (POMG) and combines Actor-Critic RL (PPO with GAE) with semi-asynchronous federated learning:

  • Federate the Critic, Specialize the Actor: Each node maintains a local actor network tuned to its specific hardware and immediate neighbors. Only the critic networks—which predict system-wide value—are shared and aggregated globally. This preserves localized decision-making while aligning agents toward a shared global objective.
  • Buffered Semi-Asynchronous Aggregation (Adapted FedBuff): Rather than waiting for every node to finish, a Global Manager (GM) maintains a buffer of critic updates. Slower agents do not block faster agents, and newly arriving updates from an agent overwrite older ones. Once $K$ distinct agent updates accumulate, the GM computes a weighted average (weighted by each agent's local steps) and redistributes the global critic.
  • Padded, Normalized State Representation: Observations are standardized with normalized placeholders (e.g., -1 for absent neighbors), ensuring consistent critic input dimensions despite topological variation.
  • Network-Aware FL Simulation: The authors extended the PeersimGym simulator with an FLManager module that accurately models the network overhead, packet size, and transmission delays of exchanging model parameters over edge links.
+-----------------------------------------------------------+
| Local Agent (Node i)                                      |
|  - Actor: Local PPO policy (neighborhood actions)         |
|  - Critic: Periodically pushes updates to Global Manager  |
+-----------------------------+-----------------------------+
                              | Asynchronous updates (Dw)
                              v
             +---------------------------------+
             | Global Manager (Buffer size K)  |
             | - Replaces stale agent updates  |
             | - Weighted average when k >= K  |
             | - Broadcasts new Global Critic  |
             +---------------------------------+

Datasets

The evaluation relies on simulated edge infrastructures combining realistic traces and topologies:

  • Workloads: Real-world traces from the Alibaba Cluster Trace dataset, rescaled (10% instruction reduction) to reflect edge device capacities without causing immediate overflow.
  • Topologies: Synthesized realistic edge topologies generated via Ether (hierarchical star/AoT clusters comprising Raspberry Pi 3/4/5 nodes, Intel NUC base stations, and edge servers) alongside synthetic 10-node and 15-node random graphs.

Key Insights and Ideas

  • Local Actors Prevent Catastrophic Policy Averaging: Averaging actor weights directly across heterogeneous devices severely degrades performance because optimal actions depend on localized link speeds and queues. Federating only the value function (critic) guides cooperative behavior without washing out individual node specialization.
  • Graceful Degradation Under Stragglers: In ablation experiments dropping up to 80% of client gradient updates ($s = 0.8$), FAuNO maintained virtually identical finished-task ratios (~0.76) and response times (~260 ticks), proving it gracefully degrades rather than collapses under extreme network unreliability.
  • Critic Agreement Under Non-IID Skew: Across heterogeneous clusters processing distinct workloads, FAuNO’s global critic achieved a global disagreement score approaching that of an oracle centralized critic (74.5 vs. 63.6), whereas independent MARL diverged heavily (disagreement scores > 147–313).

Key Results

  • Ether Hierarchical Topologies: FAuNO consistently achieved the highest task completion rates across varied arrival rates ($\lambda \in \{0.5, 1, 2\}$), reaching 92.0%–97.4% completion, compared to synchronous FRL baseline SCOF (66.6%–94.3%) and Least Queues (90.9%–94.8%).
  • Latency vs. Throughput Trade-off: While SCOF recorded low response times at heavy loads, it did so by failing and dropping over 30% of incoming tasks. FAuNO delivered balanced response times (170–262 ticks) while preserving task delivery.
  • Architectural Ablations: Critic-only federation consistently outperformed variants that federated the actor or both actor and critic (e.g., at $\lambda=2$, FAuNO achieved a 0.926 completion ratio vs. 0.736 for federated-actor and 0.848 for actor-critic).

Limitations and Future Work

  • Centralized Orchestrator Bottleneck: The framework relies on a single Global Manager to buffer and aggregate updates, introducing a potential single point of failure and scalability limits for large-scale deployments.
  • Cooperative & Honest Node Assumption: The current formulation does not guard against adversarial or Byzantine nodes, nor does it incorporate differential privacy.
  • Simulation-Only: The framework has been validated solely within simulation (PeersimGym) under static network links; mobile nodes, dynamic wireless fading, and energy consumption tradeoffs remain open topics for future physical-testbed deployment.

Summary updated

Abstract

Edge computing addresses the growing data demands of connected‐device networks by placing computational resources closer to end users through decentralized infrastructures. This decentralization challenges traditional, fully centralized orchestration, which suffers from latency and resource bottlenecks. We present \textbf{FAuNO}---\emph{Federated Asynchronous Network Orchestrator}---a buffered, asynchronous \emph{federated reinforcement-learning} (FRL) framework for decentralized task offloading in edge systems. FAuNO adopts an actor–critic architecture in which local actors learn node-specific dynamics and peer interactions, while a federated critic aggregates experience across agents to encourage efficient cooperation and improve overall system performance. Experiments in the \emph{PeersimGym} environment show that FAuNO consistently matches or exceeds heuristic and federated multi-agent RL baselines in reducing task loss and latency, underscoring its adaptability to dynamic edge-computing scenarios.
BibTeX
@misc{metelo2025fauno,
  author = {Frederico Metelo and Alexandre Oliveira and Stevo Rackovic and Pedro Ákos Costa and Claudia Soares},
  title = {{FAuNO: Semi-Asynchronous Federated Reinforcement Learning Framework for Task Offloading in Edge Systems}},
  year = {2025},
  eprint = {2506.02668},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.02668},
}

Loading related content...

Figure