Learning Decentralized Flocking Controllers with Spatio-Temporal Graph Neural Network

Siji Chen, Yanshen Sun, Peihan Li, Lifeng Zhou, Chang-Tien Lu

Abstract

Recently a line of researches has delved the use of graph neural networks (GNNs) for decentralized control in swarm robotics. However, it has been observed that relying solely on the states of immediate neighbors is insufficient to imitate a centralized control policy. To address this limitation, prior studies proposed incorporating $L$-hop delayed states into the computation. While this approach shows promise, it can lead to a lack of consensus among distant flock members and the formation of small clusters, consequently resulting in the failure of cohesive flocking behaviors. Instead, our approach leverages spatiotemporal GNN, named STGNN that encompasses both spatial and temporal expansions. The spatial expansion collects delayed states from distant neighbors, while the temporal expansion incorporates previous states from immediate neighbors. The broader and more comprehensive information gathered from both expansions results in more effective and accurate predictions. We develop an expert algorithm for controlling a swarm of robots and employ imitation learning to train our decentralized STGNN model based on the expert algorithm. We simulate the proposed STGNN approach in various settings, demonstrating its decentralized capacity to emulate the global expert algorithm. Further, we implemented our approach to achieve cohesive flocking, leader following and obstacle avoidance by a group of Crazyflie drones. The performance of STGNN underscores its potential as an effective and reliable approach for achieving cohesive flocking, leader following and obstacle avoidance tasks.

Introduction

In swarm robotics, "flocking" represents a fundamental coordination challenge. The objective is for a group of agents to move together cohesively, follow a leader, and avoid obstacles. While centralized controllers can achieve this using global state information, real-world applications require decentralized solutions where each robot relies only on local sensing and communication.

Recent efforts have utilized Graph Neural Networks (GNNs) to bridge this gap. However, standard GNNs often fail to achieve global consensus, leading to fragmented swarms. This paper introduces the Spatio-Temporal Graph Neural Network (STGNN), a decentralized controller that mimics complex centralized flocking behaviors by integrating both spatial and temporal information.

Methodology

The core innovation of STGNN lies in its dual-expansion architecture, which processes local observations through two parallel pipelines to capture a more holistic view of the swarm's state.

The STGNN Architecture

The model employs Imitation Learning to learn from a centralized expert algorithm. The STGNN architecture consists of:

  • Spatial Expansion: This pipeline aggregates $L$-hop delayed states, allowing an agent to indirectly "see" distant neighbors by processing information propagated through the network over time.
  • Temporal Expansion: This pipeline focuses on the historical states of immediate neighbors, capturing the local evolution of movement.
  • Transformer-based Fusion: Both pipelines utilize Transformer encoders to fuse information across timestamps. The resulting features are concatenated and passed through a Multi-Layer Perceptron (MLP) to predict control signals (acceleration).
The STGNN architecture, illustrating the parallel spatial and temporal expansion pipelines used to fuse multi-hop and historical neighbor data.

Centralized Expert Algorithm

The expert algorithm provides the training signal, integrating potential fields for cohesion and alignment, peer-to-peer guidance for leader following, and a novel "$\beta$-robot" approach for obstacle avoidance.

Experimental Results

The STGNN model was evaluated in both simulated environments and real-world drone experiments.

Simulation Performance

The model was trained on a swarm of 20 robots and tested for scalability on swarms of up to 50 robots. Results indicate that STGNN significantly outperforms spatial-only (DGNN) and temporal-only (TGNN) variants.

Performance comparison of various models. STGNN L3 achieves a 100% completion rate and the lowest Mean Absolute Error (MAE) compared to other decentralized baselines.

Key findings include:

  • High Fidelity: STGNN with a history length of 3 ($L=3$) achieved a 100% completion rate, matching the expert's performance.
  • Precision: The Mean Absolute Error (MAE) for STGNN $L3$ was 1.71, notably lower than DGNN (2.83) and TGNN (2.11).
  • Cohesion: A near-zero velocity variance ($V=0.01$) demonstrates superior alignment across the swarm.

Real-World Validation

The controller was successfully deployed on a fleet of six Crazyflie 2.1 drones. Despite being trained in simulation, the STGNN-based controller enabled the drones to maintain tight formations and navigate obstacle courses in real-time, proving its robustness and transferability.

Conclusion

The STGNN framework demonstrates that combining spatial and temporal information is crucial for effective decentralized swarm control. By leveraging Transformer-based fusion of multi-hop and historical data, STGNN achieves near-expert performance in complex flocking tasks. Future work will explore extending this architecture to more diverse multi-agent challenges such as collaborative exploration and target tracking.

Summary updated

Abstract

Recently a line of researches has delved the use of graph neural networks (GNNs) for decentralized control in swarm robotics. However, it has been observed that relying solely on the states of immediate neighbors is insufficient to imitate a centralized control policy. To address this limitation, prior studies proposed incorporating $L$-hop delayed states into the computation. While this approach shows promise, it can lead to a lack of consensus among distant flock members and the formation of small clusters, consequently resulting in the failure of cohesive flocking behaviors. Instead, our approach leverages spatiotemporal GNN, named STGNN that encompasses both spatial and temporal expansions. The spatial expansion collects delayed states from distant neighbors, while the temporal expansion incorporates previous states from immediate neighbors. The broader and more comprehensive information gathered from both expansions results in more effective and accurate predictions. We develop an expert algorithm for controlling a swarm of robots and employ imitation learning to train our decentralized STGNN model based on the expert algorithm. We simulate the proposed STGNN approach in various settings, demonstrating its decentralized capacity to emulate the global expert algorithm. Further, we implemented our approach to achieve cohesive flocking, leader following and obstacle avoidance by a group of Crazyflie drones. The performance of STGNN underscores its potential as an effective and reliable approach for achieving cohesive flocking, leader following and obstacle avoidance tasks.
BibTeX
@inproceedings{chen2024learning,
  author = {Siji Chen and Yanshen Sun and Peihan Li and Lifeng Zhou and Chang-Tien Lu},
  title = {{Learning Decentralized Flocking Controllers with Spatio-Temporal Graph Neural Network}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  doi = {10.1109/icra57147.2024.10610627},
}
Figure