VGAI: End-to-End Learning of Vision-Based Decentralized Controllers for Robot Swarms

Ting-Kuei Hu, Fernando Gama, Tianlong Chen, Zhangyang Wang, Alejandro Ribeiro, Brian M. SadlerICASSP 2021 · DOI · OpenReview

TL;DR

This paper introduces VGAI, a novel end-to-end framework that learns decentralized robot swarm controllers solely from raw visual inputs. By jointly training Convolutional Neural Networks (CNNs) for visual perception and Graph Neural Networks (GNNs) for communication, VGAI enables large swarms to coordinate effectively using only local information, achieving successful flocking for up to 75 drones.

What Problem Does This Paper Address?

Coordinating large swarms of robots, especially for tasks like aerial surveillance or disaster response, presents a significant challenge. Traditional approaches often rely on centralized control, which is vulnerable to single points of failure and communication bottlenecks. While decentralized control offers robustness, it's difficult to achieve complex coordinated behaviors using only local sensory information and limited communication. Specifically, this paper tackles the problem of enabling robots to learn decentralized controllers that can interpret their surroundings through raw visual data and communicate effectively with neighbors to achieve a global objective, such as flocking, without relying on external localization systems like GPS. The core challenge lies in bridging the "semantic gap" between raw visual perception and actionable control decisions in a decentralized, scalable manner.

Methods

The proposed framework, VGAI (Vision-based Graph Aggregation and Inference), is an end-to-end learning system composed of three stages:

  1. Visual State Estimator (CNN): Each robot uses a Convolutional Neural Network (CNN) to process its raw visual input (a panorama of four camera views). This CNN extracts a compact "visual state feature" that is efficient for transmission. The CNN architecture is based on residual blocks and is designed to preserve spatial information relevant for flocking.
  2. Local Graph Aggregation (GNN): A Graph Neural Network (GNN) is employed to manage communication and information aggregation. This GNN operates on a graph representing the swarm, where nodes are agents and edges represent communication links (based on proximity). The GNN uses a "delayed aggregation" mechanism, where agents iteratively exchange their visual state features with their immediate neighbors. This process allows information to propagate across multiple "hops" (communication steps), effectively incorporating information from farther away neighbors, albeit with delays.
  3. Action Inference (GNN): The aggregated information from the GNN is then used by the same GNN to predict the robot's next action (acceleration command). The GNN is trained to map the combined local visual features and aggregated neighbor information to an optimal control action.

Crucially, the CNN and GNN are trained jointly using imitation learning. This means the system learns to extract visual features that are most useful for the GNN to make good control decisions, and the GNN learns how to best utilize these features and communicate with neighbors. This joint training is performed end-to-end using backpropagation through time (BPTT).

Datasets

The paper does not explicitly mention using pre-existing datasets for training. Instead, the experiments are conducted in the Microsoft Airsim Simulation environment. This environment allows for the rendering of realistic visual inputs (omnidirectional images from four cameras) and the simulation of swarm dynamics. The training data is generated by collecting trajectories of optimal actions from a centralized controller and using these as ground truth for imitation learning.

Key Insights and Ideas

  • Bridging Vision and Communication: The most significant insight is the successful integration of learning from raw visual inputs and learning communication strategies within a single, end-to-end decentralized framework. This addresses a key limitation in prior work that focused on one aspect or the other.
  • Scalability through GNNs: The use of GNNs, with their inherent permutation equivariance and weight-sharing properties, is critical for scaling to large swarms. The learned controller is agnostic to the number of agents, allowing it to be trained on smaller teams and deployed on larger ones.
  • Local Information Suffices: The framework demonstrates that complex coordinated behaviors like flocking can be achieved using only local visual observations and sparse, local communication, without relying on global state information.
  • Learned Communication is Key: The joint training of the CNN and GNN allows the system to learn what visual features are important to communicate and how to best aggregate information from neighbors to achieve the desired global behavior.

Key Results

  • Successful Decentralized Flocking: VGAI achieved successful decentralized flocking for swarms of up to 75 drones.
  • Performance Comparison: VGAI's performance (measured by relative velocity variation cost) was comparable to, and in some cases better than, a decentralized controller (DAGNN) that was given ideal neighbor information (not derived from vision). It also significantly outperformed a simple local heuristic controller.
  • Hyperparameter Sensitivity: The paper shows that increasing the visual state feature dimension (F) and the number of communication exchanges (K-1) generally improves VGAI's performance, up to a point.
  • Robustness: VGAI demonstrated robustness to variations in initial velocity and communication radius. Higher initial speeds made control harder, while larger communication radii improved performance.
  • Scalability: A controller trained on 50 agents could successfully flock teams of up to 75 agents, highlighting its generalization capabilities.

Limitations and Future Work

  • Imitation Learning Dependency: The current framework relies on imitation learning from a centralized controller. This means its performance is bounded by the quality of the "expert" policy.
  • Numerical Stability: The paper mentions saturating acceleration commands to improve numerical stability during training, suggesting that extreme control actions can be an issue.
  • Future Work: The authors suggest exploring other end-to-end visual learning frameworks to further improve cohesion behaviors. They also implicitly point towards investigating more complex swarm tasks beyond flocking.

Summary updated

Abstract

Decentralized coordination of a robot swarm requires addressing the tension between local perceptions and actions, and the accomplishment of a global objective. In this work, we propose to learn decentralized controllers based solely on raw visual inputs. For the first time, this integrates the learning of two key components: communication and visual perception, in one end-to-end framework. More specifically, we consider that each robot has access to a visual perception of the immediate surroundings, and communication capabilities to transmit and receive messages from other neighboring robots. Our proposed learning framework combines a convolutional neural network (CNN) for each robot to extract messages from the visual inputs, and a graph neural network (GNN) over the entire swarm to transmit, receive and process these messages in order to decide on actions. The use of a GNN and locally-run CNNs results naturally in a decentralized controller. We jointly train the CNNs and the GNN so that each robot learns to extract messages from the images that are adequate for the team as a whole. Our experiments demonstrate the proposed architecture in the problem of drone flocking and show its promising performance and scalability, e.g., achieving successful decentralized flocking for large-sized swarms.
BibTeX
@inproceedings{hu2021vgai,
  author = {Ting-Kuei Hu and Fernando Gama and Tianlong Chen and Zhangyang Wang and Alejandro Ribeiro and Brian M. Sadler},
  title = {{VGAI: End-to-End Learning of Vision-Based Decentralized Controllers for Robot Swarms}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9414219},
}
Figure