A Program to Build E(N)-Equivariant Steerable CNNs

Gabriele Cesa, Leon Lang, Maurice WeilerICLR 2022 · OpenReview

Executive Summary

This paper introduces a general-purpose mathematical framework and software library for building "steerable" Convolutional Neural Networks (CNNs) that are equivariant to any symmetry group (like rotations and reflections) in 2D or 3D. By generalizing the Wigner-Eckart theorem, the authors provide a unified way to design networks that perfectly respect the symmetries of the underlying data, leading to better performance and higher data efficiency.

The Challenge of Equivariance and Aliasing

In deep learning, "equivariance" means that if you transform the input (e.g., rotate an image), the network's internal features transform in a predictable, corresponding way. While standard CNNs are naturally equivariant to translations, making them equivariant to rotations or reflections is mathematically complex.

Previous "steerable" CNNs were often ad-hoc. Researchers had to manually derive complex mathematical constraints for every specific symmetry group they wanted to support (e.g., one paper for 90-degree rotations, another for full 3D rotations).

Continuous vs. discrete group orbits on a pixel grid.

Furthermore, existing methods often struggled with aliasing. When you discretize a continuous rotation onto a pixel grid, you introduce errors. Prior work tried to solve this by defining filters on independent "orbits" (concentric rings), but patching these rings together into a smooth filter was difficult and often led to artifacts.

A Unified Mathematical Framework

The authors propose a systematic "program" to build these networks for any compact group $G$. Their key technical contributions include:

  1. Generalized Wigner-Eckart Theorem: They extend a core theorem from quantum mechanics to characterize equivariant kernels over any space, not just specific orbits. This allows them to define filters using a global, band-limited basis (like circular or spherical harmonics) from the start, mitigating aliasing issues.
  2. Group Restriction: Instead of starting from scratch for every group, they suggest starting with a basis for the largest possible group—the orthogonal group $O(n)$, which includes all rotations and reflections. They then use representation theory to "restrict" this basis to any desired subgroup (like the symmetries of a cube or a cylinder).
Constructing equivariant kernels using Clebsch-Gordan tensor products.
  1. Numerical Implementation: They provide an algorithm to compute these complex mathematical objects numerically, making it practical to implement equivariance for exotic groups without manual derivation.
  2. Regular Non-linearities: For 3D data, they introduce efficient versions of standard non-linearities (like ReLU) that work within the steerable framework by sampling features on the group and then transforming them back.

The authors released their work as the escnn library, a general-purpose tool for PyTorch.

Key Insights and Empirical Validation

The framework was evaluated on three distinct types of data: Rotated MNIST (2D), Rotated ModelNet10 (3D voxelized shapes), and Ligand Binding Affinity (3D molecular data).

  • Symmetry Matching is Key: A major finding is that "more symmetry isn't always better." If your data only has axial symmetry (e.g., objects that are always upright but can rotate horizontally), a model that assumes full 3D rotation symmetry might be too constrained. On a version of ModelNet10 with only azimuthal (horizontal) symmetry, an $O(2)$-equivariant model outperformed a full $SO(3)$ model (92.3% vs 89.8%).
  • Discrete vs. Continuous: For data on a grid (voxels), discrete symmetry groups (like the symmetries of Platonic solids) often perform better than continuous ones because they better align with the underlying grid structure. The authors' icosahedral-equivariant model (60 symmetries) achieved 90.0% accuracy on ModelNet10, significantly outperforming standard CNNs (82.5%).
  • Molecular Data (LBA): Their $SO(3)$ model with "regular non-linearities" achieved a state-of-the-art Pearson correlation of 0.589, while being computationally equivalent to a standard 3D CNN.

Limitations and Future Work

While highly robust, the framework has a few limitations. The current method parameterizes filters on a compact ball (a disk in 2D or a sphere in 3D). This is slightly more restrictive than the standard square or cubic filters used in traditional CNNs, which can exploit the "corners" of the kernel. Additionally, while the paper uses Gaussian radial profiles and spherical harmonics, the choice of the initial basis is a design decision that could be further optimized. Finally, some steerable operations still carry a small computational overhead compared to highly optimized standard convolution implementations.

Summary updated

Abstract

Equivariance is becoming an increasingly popular design choice to build data efficient neural networks by exploiting prior knowledge about the symmetries of the problem at hand. Euclidean steerable...
BibTeX
@inproceedings{cesa2022program,
  author = {Gabriele Cesa and Leon Lang and Maurice Weiler},
  title = {{A Program to Build E(N)-Equivariant Steerable CNNs}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=0qf71T2bZGm},
}
Figure