Learning Physical Dynamics with Subequivariant Graph Neural Networks

Jiaqi Han, Wenbing Huang, Hengbo Ma, Jiachen Li, Joshua B. Tenenbaum, Chuang Gan

Simulating the physical world is a cornerstone of robotics, engineering, and computer graphics. While Graph Neural Networks (GNNs) have become the standard tool for learning these dynamics from data, they often struggle with a fundamental tension: symmetry. If you train a model to watch dominoes fall from left to right, will it understand what happens if you rotate the whole table 90 degrees?

In their NeurIPS 2022 paper, "Learning Physical Dynamics with Subequivariant Graph Neural Networks," Jiaqi Han and colleagues introduce a framework that finds the "Goldilocks zone" of symmetry—respecting the laws of physics without being blinded by them.

The Symmetry Dilemma

In physics, equivariance means that if you transform the input (e.g., rotate a scene), the output (the predicted motion) should transform in the exact same way. Current GNN simulators typically fall into two traps:

  1. Insufficient Symmetry: Standard GNNs (like GNS) treat coordinates as raw numbers. They often overfit to the specific orientations seen in training data and fail to generalize to rotated environments.
  2. Excessive Symmetry: "Equivariant" GNNs (like EGNN) enforce full $E(3)$ symmetry (rotations and reflections in all directions). However, we live in a world with gravity. Gravity defines a "down" direction, breaking full symmetry. If a model is strictly $E(3)$-equivariant, it struggles to learn that things always fall toward the floor, regardless of how the camera is tilted.

Furthermore, most particle-based simulators treat all particles identically, ignoring the fact that particles belong to specific objects with distinct shapes and properties, such as a rigid block versus a fluid.

Subequivariant Graph Neural Networks (SGNN)

The authors propose Subequivariant Graph Neural Networks (SGNN) to bridge these gaps. Instead of full $E(3)$ symmetry, they target the subgroup of symmetries that remain valid in the presence of gravity: rotations and reflections around the vertical axis ($O_g(3)$). They mathematically prove that their proposed subequivariant functions are "universally expressive," meaning they can approximate any physical function that respects this specific symmetry.

Comparison of message passing schemes across different GNN architectures. SGNN incorporates the gravity vector g and object centers c to achieve subequivariance.

As shown in the comparison above, SGNN explicitly incorporates the gravity vector $\vec{g}$ and object-level information (like the center of mass $\vec{c}_{o(i)}$) into the particle-level communication. This allows the model to "understand" the geometry of the objects it is simulating, which is vital for capturing effects like torque.

Hierarchical Object-aware Message Passing

To handle different types of physical constraints, the simulation operates in three distinct stages of hierarchical message passing:

  • Inter-object: Particles from different objects exchange momentum during collisions.
  • Object-level: A high-level summary of how objects interact as wholes.
  • Inner-object: Particles within the same object communicate to maintain shape and rigid constraints.
The hierarchical message passing architecture of SGNN, showing inter-object, object-level, and inner-object communication stages.

This hierarchical approach ensures that the model respects "objectness." Explicitly labeling which particles belong to which object significantly improves the model's ability to handle complex shapes without having to re-learn the concept of an object from scratch.

Key Results and Generalization

The model was evaluated on two challenging benchmarks: Physion (8 diverse scenarios like Dominoes, Collide, and Link) and RigidFall (cubes falling under varying gravitational strengths).

  • Accuracy: SGNN achieved a 3% enhancement in contact prediction accuracy across Physion scenarios and a 2x lower rollout error (MSE) on RigidFall compared to state-of-the-art simulators like GNS and DPI.
  • Generalization: When tested on scenes rotated around the gravity axis, SGNN maintained perfect performance, while non-equivariant models saw their accuracy plummet. For example, GNS accuracy dropped from 78.6% to 53.3% on the Dominoes task.
  • Robustness: Even when the "table" was tilted (rotating the scene around a non-gravity axis), SGNN generalized better than competitors, correctly predicting that dominoes would slide down the incline.

Conclusion

SGNN demonstrates that "more symmetry" isn't always better. By relaxing $E(3)$ to $O_g(3)$, the model gains the flexibility to learn the effect of gravity while retaining the ability to generalize across horizontal rotations. While powerful, the model currently relies on particle-based representations and does not explicitly handle self-contact within deformable objects. Future work may involve integrating SGNN with visual encoders to learn directly from pixels or incorporating stronger physical priors like Hamiltonian mechanics to ensure long-term energy conservation.

Summary updated

Abstract

Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. However, they still encounter several challenges: 1) Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization and should be incorporated into the model design. Existing simulators either consider insufficient symmetry, or enforce excessive equivariance in practice when symmetry is partially broken by gravity. 2) Objects in the physical world possess diverse shapes, sizes, and properties, which should be appropriately processed by the model. To tackle these difficulties, we propose a novel backbone, called Subequivariant Graph Neural Network, which 1) relaxes equivariance to subequivariance by considering external fields like gravity, where the universal approximation ability holds theoretically; 2) introduces a new subequivariant object-aware message passing for learning physical interactions between multiple objects of various shapes in particle-based representation; 3) operates in a hierarchical fashion, allowing for modeling long-range and complex interactions. Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2$\times$ lower rollout MSE on RigidFall compared with state-of-the-art GNN simulators, while exhibiting strong generalization and data efficiency.
BibTeX
@inproceedings{han2022learning,
  author = {Jiaqi Han and Wenbing Huang and Hengbo Ma and Jiachen Li and Joshua B. Tenenbaum and Chuang Gan},
  title = {{Learning Physical Dynamics with Subequivariant Graph Neural Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=siG_S8mUWxf},
}
Figure