In geometric deep learning, baking physical laws directly into a network's architecture is a powerful strategy. By enforcing equivariance—ensuring that if an input (like a 3D molecule or a point cloud) rotates, its internal representations rotate accordingly—models gain immense data efficiency and physical consistency.
However, hard-coding strict mathematical equivariance comes with a hidden cost: a severe penalty to the model's expressivity and architectural flexibility. This tension is known as the equivariance-expressivity trade-off. When we force a network to respect strict geometric symmetries, we inadvertently restrict its ability to learn complex, non-symmetric patterns, optimize efficiently, or leverage high-capacity standard backbones.
To build better models for physical systems, molecular dynamics, and computer vision, researchers are shifting away from rigid, "equivariant-by-design" architectures toward methods that dynamically relax, break, or bypass these symmetry constraints.
The Mathematical Cage of Strict Equivariance
Why does enforcing a symmetry group like $SO(3)$ (3D rotations) or $E(3)$ (rotations and translations) hurt a model's capacity to learn?
To guarantee that a network is strictly equivariant, its internal mathematical operations must be heavily restricted. As Shitong Luo et al. point out in Equivariant Point Cloud Analysis via Learning Orientations for Message Passing, many standard equivariant frameworks limit transformations to linear combinations, banning the highly expressive non-linear functions that give deep neural networks their power. Similarly, Lexiang Hu et al. demonstrate in Incorporating Arbitrary Matrix Group Equivariance into KANs that forcing non-linear components to respect symmetry requires restrictive gating mechanisms, capping the network's theoretical function-approximation power compared to unconstrained counterparts.
Furthermore, strict equivariance drastically reduces the degrees of freedom in a network's weight matrices. As explored by Bahar Azari et al. in Equivariant Deep Dynamical Model for Motion Prediction, rigid symmetry constraints force extensive parameter sharing. This shrinks the searchable parameter space to a tiny fraction of possible weight configurations. The authors of Improving Equivariant Model Training via Constraint Relaxation note that this highly restricted space makes optimization notoriously difficult, leading to poor convergence and extreme sensitivity to hyperparameters.
When Symmetry Blinds the Network
Beyond optimization difficulties, strict symmetry can actively prevent a network from learning critical features of the data.
Consider a perfectly flat, planar molecule. As Jonas Elsborg et al. show in ELECTRA: A Symmetry-breaking Cartesian Network for Charge Density Prediction with Floating Orbitals, a standard, strictly equivariant network can become mathematically incapable of predicting coordinates or features that exist outside that molecular plane. By breaking local symmetry constraints while maintaining global rotational equivariance, their architecture recovers the expressivity needed for accurate physical predictions.
A similar issue arises in image processing and computer vision. In Rotation-Equivariant Self-Supervised Method in Image Denoising, Hanze Liu et al. explain that hard rotational constraints limit a network's capacity to model intricate, non-symmetric textures. If a model must treat all orientations identically, it cannot exploit fixed spatial biases or capture fine-grained asymmetric details.
This problem is so pronounced that in certain domains, like 3D human pose estimation, Cuong Le demonstrates in On the Role of Rotation Equivariance in Monocular 3D Human Pose Estimation that simple data augmentation on a "vanilla," unconstrained network often outperforms complex equivariant-by-design architectures. The unconstrained model retains the freedom to learn complex, non-linear mappings that rigid geometric constraints block.
Escaping the Trade-off via Canonicalization
If hard-coding equivariance into every layer cripples expressivity, how can we still benefit from geometric priors? One prominent solution is canonicalization (or frame averaging).
Instead of building a specialized equivariant network from scratch, we can use a lightweight, equivariant module to compute a standardized orientation—a "canonical frame"—for the input data. Once the data is aligned to this frame, it can be passed into a standard, unconstrained, high-capacity backbone like a vanilla Transformer or CNN.
Strictly Equivariant Network:
Input ──> [Equivariant Layer (Restricted Weights)] ──> [Equivariant Layer] ──> Output
Canonicalization Approach:
Input ──> [Canonicalization Module] ──> Standardized Frame ──> [High-Capacity Backbone (Unconstrained)] ──> Output
The authors of Equivariance with Learned Canonicalization Functions demonstrate that this approach completely sidesteps the expressivity penalty, allowing researchers to deploy state-of-the-art standard backbones that are faster and more flexible than traditional equivariant models.
This architecture-agnostic strategy is highly versatile:
- Inertial Navigation: Royina Karegoudra Jayanth et al. use this in EqNIO: Subequivariant Neural Inertial Odometry to map messy IMU data into a gravity-aligned canonical frame before processing it with a high-capacity backbone.
- Point Clouds: Omri Puny et al. formalize this concept in Frame Averaging for Invariant and Equivariant Network Design, providing a systematic framework to make any off-the-shelf architecture invariant or equivariant without altering its internal layers.
- Materials Science: Sergey Pozdnyakov et al. apply an ensemble version of this concept in Smooth, exact rotational symmetrization for deep learning on point clouds, wrapping a highly expressive, unconstrained "Point Edge Transformer" to achieve state-of-the-art accuracy while maintaining exact physical symmetries.
Dynamic Relaxation and Quasi-Equivariance
Another frontier in navigating this trade-off is symmetry relaxation, where the strictness of the geometric constraints is modulated dynamically or learned directly from data.
In Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data, Stefanos Pertigkiozoglou et al. introduce a framework that automatically tunes the strictness of symmetry constraints on a per-layer basis. This allows early layers to leverage the data efficiency of strict equivariance, while deeper layers relax those constraints to capture complex, non-symmetric nuances.
Similarly, the work in A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs parameterizes symmetry as a learnable distribution over transformation groups. This enables the network to discover the optimal balance between the data efficiency of full equivariance and the discriminative power needed to distinguish non-symmetric features (such as mirrored orientations).
When dealing with complex spaces where strict transformations are too restrictive, researchers use quasi-equivariance. For instance, the authors of Quasi-Equivariant Metanetworks relax strict weight-space symmetries to boost model expressivity while preserving functional identity. In parallel, Luke Thompson et al. implement a quasi-equivariant design in ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamics, bypassing the steep optimization and computational costs of layer-by-layer equivariance to achieve zero-shot generalization across diverse molecular trajectories.
By treating symmetry as a flexible spectrum rather than an all-or-nothing architectural constraint, modern geometric deep learning successfully balances the statistical benefits of physical priors with the raw representational capacity of unconstrained neural networks.