My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control

Vitaly Kurin, Maximilian Igl, Tim Rocktäschel, Wendelin Boehmer, Shimon Whiteson

Abstract

In robot reinforcement learning, multitask control across varying morphologies typically employs Graph Neural Networks (GNNs) to encode physical skeletal structures. This paper challenges the necessity of this "morphological inductive bias." Through extensive ablation studies, we demonstrate that explicitly encoding a robot's physical skeleton does not necessarily improve performance and can even hinder learning. We propose AMORPHEUS, a Transformer-based architecture that treats the robot as a fully connected graph. By allowing global communication between limbs via self-attention, AMORPHEUS achieves state-of-the-art results across several MuJoCo benchmarks, proving that dynamic global coordination is more effective than mimicking static physical connectivity.

Introduction

A central goal in robotics is developing a single "brain" capable of controlling diverse "bodies." This multitask incompatible control problem is often addressed using GNNs, where nodes represent limbs and edges represent physical joints. The underlying assumption is that this structure helps the network understand force propagation. However, recent findings suggest that this physical structure might act as a "cage," restricting information flow and complicating the learning process. This paper investigates whether the morphological inductive bias is truly beneficial or if it serves as a bottleneck.

Methods

The investigation proceeded in two phases: a rigorous ablation of existing GNN-based methods and the development of a new architecture.

Ablation of Morphological Bias

Researchers tested existing models like NerveNet and SMP by replacing their skeleton-based graphs with arbitrary topologies:

  • Star Graphs: All limbs connect only to a central torso.
  • Line Graphs: Limbs are connected in a random linear sequence.
  • Fully Connected Graphs: Every limb connects to every other limb.

The AMORPHEUS Architecture

To overcome the limitations of sparse GNNs, the authors proposed AMORPHEUS, a model leveraging the Transformer architecture. Unlike GNNs that require multiple message-passing steps for distant limbs to communicate, AMORPHEUS enables instantaneous global communication.

The AMORPHEUS architecture uses a Transformer to enable global communication between limbs.

The architecture consists of three main components:

  1. Encoder: Maps individual limb features (e.g., position, velocity) into a latent space.
  2. Transformer: Employs $k$ layers of self-attention, allowing each limb to attend to all others, facilitating state-dependent coordination.
  3. Decoder: Translates the processed representations into specific motor torques for each joint.

Results

The evaluation spanned several MuJoCo-based multitask suites, including Walker++, Humanoid++, and a classic "Walkers" benchmark.

Morphology as a Bottleneck

The ablation studies revealed a surprising trend: models using "star" or "line" graphs often matched or exceeded the performance of those using the actual physical morphology. In sparse physical graphs, information from a foot might take several "hops" to reach the torso, leading to "over-squashing" and gradient signal loss in GNNs.

Performance comparison across different graph topologies for SMP and NerveNet shows that physical morphology is often not the optimal inductive bias.

AMORPHEUS Performance

AMORPHEUS consistently outperformed previous state-of-the-art methods like SMP. It demonstrated superior sample efficiency and stability, particularly in complex environments like Cheetah++, where SMP often struggled. Analysis of attention masks showed that AMORPHEUS learns to prioritize communication between specific limbs based on the current phase of the gait, such as coordinating limbs just before ground contact.

Discussion and Conclusion

The findings suggest that a limb's local features implicitly provide sufficient information about its role in the body, making hard-coded edges redundant. By moving from fixed physical structures to dynamic, attention-based communication, AMORPHEUS avoids the "cage" of morphology. While the $O(N^2)$ complexity of Transformers is a consideration for systems with massive numbers of components, for standard robotic control, the benefits of global communication far outweigh the computational costs. This work paves the way for more flexible, morphology-agnostic controllers in multitask reinforcement learning.

Summary updated

BibTeX
@inproceedings{kurin2021my,
  author = {Vitaly Kurin and Maximilian Igl and Tim Rocktäschel and Wendelin Boehmer and Shimon Whiteson},
  title = {{My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=N3zUDGN5lO},
}
Figure