Heterogeneous Multi-agent Collaboration in UAV-assisted Mobile Crowdsensing Networks

Xianyang Deng, Wenshuai Liu, Yaru FuB, Qi ZhuarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces CKAN-HAPPO, a multi-agent reinforcement learning framework designed to coordinate UAVs and mobile users in crowdsensing networks. By replacing traditional neural network layers with Kolmogorov-Arnold Networks (KANs), the authors significantly improve the efficiency of joint sensing, communication, and computation, maximizing the amount of data processed in dynamic, infrastructure-limited environments.

What Problem Does This Paper Address?

Mobile Crowdsensing (MCS) leverages the sensors in our pockets (smartphones, wearables) to collect massive amounts of data for smart cities or environmental monitoring. However, relying on fixed cell towers is risky in remote areas or disaster zones. Unmanned Aerial Vehicles (UAVs) can act as mobile base stations and edge computing servers to bridge this gap.

The challenge is a "triple threat" of interdependencies:

  1. Sensing: Users need to decide how much data to collect.
  2. Communication: Users must offload data to moving UAVs over fluctuating wireless channels.
  3. Computation: Both users and UAVs must process this data within tight time and energy constraints.

Because users and UAVs are constantly moving, the network topology is always shifting. Optimizing this system is a "non-convex stochastic optimization" problem—essentially a mathematical nightmare that traditional algorithms struggle to solve in real-time.

Methods

The authors model the environment as a Partially Observable Markov Decision Process (POMDP), acknowledging that no single agent (UAV or user) has a perfect "god-view" of the entire network.

To solve this, they propose CKAN-HAPPO, which builds on Heterogeneous Agent Proximal Policy Optimization (HAPPO). The key innovations include:

  • Hybrid Actor Network: Instead of using standard Multi-Layer Perceptrons (MLPs), they combine Convolutional Neural Networks (CNNs) for spatial feature extraction with Kolmogorov-Arnold Networks (KANs). Unlike MLPs, which use fixed activation functions on nodes, KANs use learnable spline-based functions on the "edges" (weights). This allows the model to capture complex, non-linear relationships with far fewer parameters.
  • Action Space Reduction: The authors derived a closed-form mathematical expression (Theorem 1) to calculate the optimal local computation frequency. By solving this part of the problem analytically, they reduced the "workload" for the AI, leading to faster and more stable training.
  • Joint Optimization: The model simultaneously manages 3D UAV trajectories, user-UAV association, bandwidth allocation, and time-slot partitioning.

Datasets

The study relies on a high-fidelity simulation environment rather than a static dataset.

  • Environment: A 1000m × 1000m square area.
  • Agents: 20 mobile users following a Gauss-Markov mobility model and 5–11 UAVs.
  • Channel Model: Rician fading to simulate realistic air-to-ground communication.
  • Benchmarks: The proposed method was compared against MLP-HAPPO (the industry standard) and CNN-HAPPO (an ablation study to prove the value of the KAN component).

Key Insights and Ideas

  • The Power of KANs: The most significant "aha moment" is that KANs are exceptionally well-suited for the high-dimensional continuous action spaces found in robotics and aerospace. They model the "physics" of the state-action dependencies more accurately than traditional deep learning.
  • Analytical Assistance: The paper demonstrates that Reinforcement Learning (RL) shouldn't always be a "black box." By using a mathematical theorem to handle the CPU frequency allocation, the RL agent can focus its "attention" on harder tasks like trajectory planning.
  • The Bandwidth Bottleneck: The results show a "saturation effect." While adding more UAVs or increasing their compute power helps, the system eventually hits a ceiling imposed by the fixed communication bandwidth. This highlights that compute power alone isn't a silver bullet for MCS.

Key Results

  • Superior Convergence: CKAN-HAPPO reached a stable, high-reward state significantly faster than the MLP-based benchmarks.
  • Data Throughput: The proposed method achieved a marked increase in the total bits of sensing data processed. For instance, in a 5-UAV scenario, it outperformed the MLP-HAPPO baseline by a visible margin in total collected data.
  • Robustness: The system maintained high performance even as the number of UAVs increased from 3 to 11, proving it can scale to larger, more complex deployments.

Limitations and Future Work

  • Fixed Bandwidth: The authors noted that fixed bandwidth limits the gains from increased UAV computation. Future work could explore dynamic spectrum sharing or higher-frequency bands (like mmWave).
  • Energy Constraints: While energy consumption is part of the reward function, the paper does not deeply explore UAVs with energy-harvesting capabilities (like solar-powered wings), which would be critical for long-term "perpetual" sensing missions.
  • Interference Management: The current model uses Orthogonal Frequency Division Multiple Access (OFDMA) to avoid interference; exploring non-orthogonal multiple access (NOMA) could further increase user density.

Summary updated

Abstract

Unmanned aerial vehicles (UAVs)-assisted mobile crowdsensing (MCS) has emerged as a promising paradigm for data collection. However, challenges such as spectrum scarcity, device heterogeneity, and user mobility hinder efficient coordination of sensing, communication, and computation. To tackle these issues, we propose a joint optimization framework that integrates time slot partition for sensing, communication, and computation phases, resource allocation, and UAV 3D trajectory planning, aiming to maximize the amount of processed sensing data. The problem is formulated as a non-convex stochastic optimization and further modeled as a partially observable Markov decision process (POMDP) that can be solved by multi-agent deep reinforcement learning (MADRL) algorithm. To overcome the limitations of conventional multi-layer perceptron (MLP) networks, we design a novel MADRL algorithm with hybrid actor network. The newly developed method is based on heterogeneous agent proximal policy optimization (HAPPO), empowered by convolutional neural networks (CNN) for feature extraction and Kolmogorov-Arnold networks (KAN) to capture structured state-action dependencies. Extensive numerical results demonstrate that our proposed method achieves significant improvements in the amount of processed sensing data when compared with other benchmarks.
BibTeX
@misc{deng2025heterogeneous,
  author = {Xianyang Deng and Wenshuai Liu and Yaru FuB and Qi Zhu},
  title = {{Heterogeneous Multi-agent Collaboration in UAV-assisted Mobile Crowdsensing Networks}},
  year = {2025},
  eprint = {2509.25261},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.25261},
  howpublished = {arXiv},
}
Figure