Imagine a digital image of a square. If you shift it 10 pixels to the right and then 20 pixels down, it ends up in the exact same position as if you had shifted it 20 pixels down first and then 10 pixels to the right. Because the order of these operations does not affect the final outcome, spatial translations are said to commute. Mathematically, they form an Abelian group.
For years, deep learning architectures have quietly relied on this commutativity assumption. From the independent latent channels of Variational Autoencoders (VAEs) to standard positional encodings in Transformers, our models excel when factors of variation can be neatly stacked or sequenced in any order.
But the physical world is rarely Abelian. Consider a 3D object: if you rotate it 90 degrees around the X-axis and then 90 degrees around the Y-axis, it lands in a completely different orientation than if you had performed those rotations in reverse.
When deep learning models encounter these non-commutative transformations (such as 3D rotations or simultaneous roto-translations), standard architectures break down. A growing body of research focuses on breaking free from the commutativity constraint, redesigning latent spaces, encodings, and kernels to natively navigate non-Abelian geometry.
Why Commutativity Fails in Latent Spaces
In representation learning, a primary goal is disentanglement—separating a dataset's underlying factors of variation (like an object's shape, position, and orientation) into distinct, interpretable components.
Standard generative models, such as InfoGAN or traditional VAEs, attempt to map these factors to independent scalar axes in a vector space. As Lin et al. demonstrate, this approach fails when the underlying transformations are non-commutative. Standard latent traversal techniques assume that you can modify one factor (e.g., pitch) linearly and independently of another (e.g., yaw). However, in a non-commutative space, performing one transformation fundamentally changes the frame of reference for the next.
Commutative (Abelian):
Step A (Translate X) ──> Step B (Translate Y) ┐
├──> Same Final State
Step B (Translate Y) ──> Step A (Translate X) ┘
Non-Commutative (Non-Abelian):
Step A (Rotate X) ──> Step B (Rotate Y) ┐
├──> Different Final States!
Step B (Rotate Y) ──> Step A (Rotate X) ┘
To resolve this, Higgins et al. provided a rigorous mathematical foundation for disentanglement based on group theory. They proved that factors governed by non-commutative transformations cannot be decomposed into independent, one-dimensional scalar components. Instead, tightly coupled non-commutative symmetries (like full 3D rotations in $SO(3)$) must be treated as a single, multi-dimensional entangled unit. The dimensionality of a truly disentangled factor is dictated by the complexity of its irreducible representation.
Unsupervised Discovery of Non-Abelian Symmetries
If non-commutative factors cannot be separated into independent lines, how can an unsupervised model discover them?
Early geometric frameworks, such as the Commutative Lie Group VAE introduced by Zhu et al., successfully used continuous Lie groups to model variations like 2D rotation and scaling. However, because they assumed the latent factors commuted, they remained blind to 3D spatial environments. Similarly, the Toroidal Subgroup Analysis framework by Cohen et al. and the biological learning models proposed by Genkin et al. noted that extending representation learning to non-Abelian groups remains a significant mathematical hurdle.
A breakthrough came with GEOMANCER, an algorithm developed by Pfau et al. that uses differential geometry to factorize non-commutative symmetries without supervision. Instead of looking for flat, linear axes, GEOMANCER analyzes the holonomy of the data manifold—tracking how tangent spaces rotate as you move along different paths.
If the transformations commute, moving along a closed loop on the manifold returns you to your exact starting orientation. If they do not commute, a discrepancy (or curvature) appears. By measuring this geometric mismatch, the model can automatically isolate complex, multi-dimensional non-Abelian factors like 3D rotations.
Commutative Loop: Non-Commutative Loop:
(Flat Space) (Curved Space)
┌───>───┐ ┌───>───┐
│ │ │ │
▲ ▼ ▲ ▼
│ │ │ │
└───<───┘ └───<───┘
Returns to original Returns with a twist
orientation. (Holonomy detected!)
To apply these geometric insights to generative modeling, Chau et al. developed Lie Group Sparse Coding (LSC). By combining sparse coding with non-commutative Lie group actions, their framework successfully separates an object's discrete identity from its continuous 3D transformations, allowing unsupervised systems to generalize to sophisticated spatial environments.
Rethinking Encodings and Kernels
The challenge of non-commutativity extends beyond latent spaces to the very core of how neural networks encode positions and compute similarities.
Positional Encodings: From RoPE to Clifford Algebras
In modern Transformers, Rotary Position Encodings (RoPE) are widely used to inject token positions by rotating query and key vectors in a 2D plane. However, RoPE assumes that the rotation generators commute, which limits its ability to handle higher-dimensional spatial data natively.
To overcome this, Sriram et al. introduced Clifford Algebraic Rotor Embeddings (CARE). By replacing standard complex numbers with Clifford algebras, CARE uses multi-dimensional "rotors" that naturally capture non-commutative spatial relationships. This allows the network to maintain shift-equivariance in 3D point clouds and images where the sequence of geometric transformations matters.
Concurrently, Schenck et al. developed STRING (Separable Translationally Invariant Position Encodings). While traditional frameworks are bound to Abelian Lie groups suitable only for Cartesian coordinates, STRING generalizes translation invariance to non-Abelian operations, enabling models to natively process full 3D rotations in $SO(3)$ for tasks like robotic manipulation.
Invariant Kernels and Optimal Transport
In kernel methods and robust architectures, researchers have historically relied on commutative groups to guarantee invariance. For instance, Bispectral Neural Networks (Sanborn et al.) and Reproducing Kernel Hilbert C*-modules (Ikeda et al.) provide powerful frameworks for learning complete invariants, but their initial implementations were restricted to compact, commutative structures. Extending these frameworks to non-Abelian groups is essential for achieving robust invariance against real-world physical transformations like 3D perspective changes.
This mathematical shift also redefines how we measure distances between data distributions. In non-commutative spaces, the lack of straight, parallel lines makes standard Optimal Transport (OT) algorithms fail. Bon et al. addressed this by developing a scalable computational framework for OT on the Lie group of roto-translations, $SE(2)$. By "lifting" 2D images into a higher-dimensional space that explicitly includes orientation, their approach enables anisotropic transport that preserves and completes continuous line structures during image interpolation.
Conclusion
The transition from commutative to non-commutative modeling represents a fundamental shift in how neural networks understand space. By moving away from the assumption that the order of variations is irrelevant, and instead embracing the non-Abelian mathematics of Lie groups, Clifford algebras, and holonomy, modern architectures can finally build representations that mirror the true geometry of our physical world.
Key Foundational Work
Learning the Irreducible Representations of Commutative Lie Groups (2014) — contributed 1 concept elements to this direction.
Learning invariant features by harnessing the aperture problem (2013) — contributed 1 concept elements to this direction.