Relaxing Symmetry Constraints for Expressive Geometric Deep Learning

115 concepts89 papers1 proposal

In geometric deep learning, baking physical laws directly into a network's architecture is a powerful strategy. By enforcing equivariance—ensuring that if an input (like a 3D molecule or a point cloud) rotates, its internal representations rotate accordingly—models gain immense data efficiency and physical consistency.

However, hard-coding strict mathematical equivariance comes with a hidden cost: a severe penalty to the model's expressivity and architectural flexibility. This tension is known as the equivariance-expressivity trade-off. When we force a network to respect strict geometric symmetries, we inadvertently restrict its ability to learn complex, non-symmetric patterns, optimize efficiently, or leverage high-capacity standard backbones.

To build better models for physical systems, molecular dynamics, and computer vision, researchers are shifting away from rigid, "equivariant-by-design" architectures toward methods that dynamically relax, break, or bypass these symmetry constraints.


The Mathematical Cage of Strict Equivariance

Why does enforcing a symmetry group like $SO(3)$ (3D rotations) or $E(3)$ (rotations and translations) hurt a model's capacity to learn?

To guarantee that a network is strictly equivariant, its internal mathematical operations must be heavily restricted. As Shitong Luo et al. point out in Equivariant Point Cloud Analysis via Learning Orientations for Message Passing, many standard equivariant frameworks limit transformations to linear combinations, banning the highly expressive non-linear functions that give deep neural networks their power. Similarly, Lexiang Hu et al. demonstrate in Incorporating Arbitrary Matrix Group Equivariance into KANs that forcing non-linear components to respect symmetry requires restrictive gating mechanisms, capping the network's theoretical function-approximation power compared to unconstrained counterparts.

Furthermore, strict equivariance drastically reduces the degrees of freedom in a network's weight matrices. As explored by Bahar Azari et al. in Equivariant Deep Dynamical Model for Motion Prediction, rigid symmetry constraints force extensive parameter sharing. This shrinks the searchable parameter space to a tiny fraction of possible weight configurations. The authors of Improving Equivariant Model Training via Constraint Relaxation note that this highly restricted space makes optimization notoriously difficult, leading to poor convergence and extreme sensitivity to hyperparameters.


When Symmetry Blinds the Network

Beyond optimization difficulties, strict symmetry can actively prevent a network from learning critical features of the data.

Consider a perfectly flat, planar molecule. As Jonas Elsborg et al. show in ELECTRA: A Symmetry-breaking Cartesian Network for Charge Density Prediction with Floating Orbitals, a standard, strictly equivariant network can become mathematically incapable of predicting coordinates or features that exist outside that molecular plane. By breaking local symmetry constraints while maintaining global rotational equivariance, their architecture recovers the expressivity needed for accurate physical predictions.

A similar issue arises in image processing and computer vision. In Rotation-Equivariant Self-Supervised Method in Image Denoising, Hanze Liu et al. explain that hard rotational constraints limit a network's capacity to model intricate, non-symmetric textures. If a model must treat all orientations identically, it cannot exploit fixed spatial biases or capture fine-grained asymmetric details.

This problem is so pronounced that in certain domains, like 3D human pose estimation, Cuong Le demonstrates in On the Role of Rotation Equivariance in Monocular 3D Human Pose Estimation that simple data augmentation on a "vanilla," unconstrained network often outperforms complex equivariant-by-design architectures. The unconstrained model retains the freedom to learn complex, non-linear mappings that rigid geometric constraints block.


Escaping the Trade-off via Canonicalization

If hard-coding equivariance into every layer cripples expressivity, how can we still benefit from geometric priors? One prominent solution is canonicalization (or frame averaging).

Instead of building a specialized equivariant network from scratch, we can use a lightweight, equivariant module to compute a standardized orientation—a "canonical frame"—for the input data. Once the data is aligned to this frame, it can be passed into a standard, unconstrained, high-capacity backbone like a vanilla Transformer or CNN.

Strictly Equivariant Network:
Input ──> [Equivariant Layer (Restricted Weights)] ──> [Equivariant Layer] ──> Output

Canonicalization Approach:
Input ──> [Canonicalization Module] ──> Standardized Frame ──> [High-Capacity Backbone (Unconstrained)] ──> Output

The authors of Equivariance with Learned Canonicalization Functions demonstrate that this approach completely sidesteps the expressivity penalty, allowing researchers to deploy state-of-the-art standard backbones that are faster and more flexible than traditional equivariant models.

This architecture-agnostic strategy is highly versatile:


Dynamic Relaxation and Quasi-Equivariance

Another frontier in navigating this trade-off is symmetry relaxation, where the strictness of the geometric constraints is modulated dynamically or learned directly from data.

In Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data, Stefanos Pertigkiozoglou et al. introduce a framework that automatically tunes the strictness of symmetry constraints on a per-layer basis. This allows early layers to leverage the data efficiency of strict equivariance, while deeper layers relax those constraints to capture complex, non-symmetric nuances.

Similarly, the work in A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs parameterizes symmetry as a learnable distribution over transformation groups. This enables the network to discover the optimal balance between the data efficiency of full equivariance and the discriminative power needed to distinguish non-symmetric features (such as mirrored orientations).

When dealing with complex spaces where strict transformations are too restrictive, researchers use quasi-equivariance. For instance, the authors of Quasi-Equivariant Metanetworks relax strict weight-space symmetries to boost model expressivity while preserving functional identity. In parallel, Luke Thompson et al. implement a quasi-equivariant design in ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamics, bypassing the steep optimization and computational costs of layer-by-layer equivariance to achieve zero-shot generalization across diverse molecular trajectories.

By treating symmetry as a flexible spectrum rather than an all-or-nothing architectural constraint, modern geometric deep learning successfully balances the statistical benefits of physical priors with the raw representational capacity of unconstrained neural networks.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_equivariance_expressivity_tradeoff_36f72d65,
  title = {equivariance-expressivity tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/equivariance-expressivity-tradeoff/rdr_36f72d65ae4f4189b9d80f01b35e5b6e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_equivariance_expressivity_tradeoff_36f72d65_v1,
  title = {equivariance-expressivity tradeoff},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/equivariance-expressivity-tradeoff/rdr_36f72d65ae4f4189b9d80f01b35e5b6e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake