Standard neural networks are remarkably good at finding patterns in sequences and grids, but they often struggle with the "physics" of 3D space. To a standard MLP, a 3D vector is just a list of three numbers; if you rotate the input, the network has no inherent reason to rotate its internal representations accordingly. Researchers have traditionally solved this using data augmentation or complex spherical harmonics, but a new direction—Clifford Group Equivariant Neural Networks—proposes a more fundamental solution: building the laws of geometry directly into the algebra of the network.
The Multivector: A Unified Geometric Language
In traditional deep learning, we treat different geometric entities—scalars (temperature), vectors (velocity), and tensors (stress)—as distinct data types. Johannes Brandstetter and David Ruhe argue that this separation is artificial. In Clifford Neural Layers for PDE Modeling, they leverage Clifford Algebra (also known as Geometric Algebra) to unify these into a single object called a multivector.
A multivector doesn't just store coordinates; it stores "grades." A grade-0 element is a scalar, grade-1 is a vector, and grade-2 is a bivector—an oriented area. By representing physical fields as multivectors, the network can naturally process the coupling between pressure (scalar) and fluid flow (vector). As Cong Liu demonstrates in Clifford Group Equivariant Simplicial Message Passing Networks, this allows models to treat higher-order structures like triangles as bivectors, preserving directional information that is typically lost when reducing geometry to simple distances or angles.
Equivariance Through the Geometric Product
The "magic" of this approach lies in the geometric product. Unlike the standard dot product (which returns a scalar) or cross product (which only works in 3D), the Clifford geometric product is a universal operation that combines two multivectors into a third.
Crucially, this product is equivariant. If you rotate the input multivectors, the result of their geometric product rotates by the same amount. David Ruhe and Patrick Forré show in Clifford Group Equivariant Neural Networks that by using the geometric product as the core interaction mechanism, the entire network becomes naturally $O(n)$ or $E(n)$ equivariant. This means the model's internal logic is independent of the coordinate system, leading to massive improvements in data efficiency.
Scaling with Geometric Algebra Transformers
While early Clifford networks were limited to small-scale simulations, the field has recently moved toward high-capacity architectures. Johann Brehmer, Pim de Haan, and Taco Cohen introduced the Geometric Algebra Transformer (GATr), which represents 3D data using 16-dimensional multivectors from Projective Geometric Algebra (PGA).
PGA is particularly powerful because it unifies points, lines, and planes into the same multivector space. In GATr, a "motor"—a specific type of multivector—can represent both a rotation and a translation. This allows the transformer to process absolute positions and relative orientations within a single mathematical framework. In Euclidean, Projective, Conformal: Choosing a Geometric Algebra for Equivariant Transformers, the authors demonstrate that this algebraic choice is not just a mathematical curiosity; it is the key to making transformers stable and performant for real-world tasks like fluid dynamics and robotics.
From Molecules to Minkowski Space
The impact of this research direction is most visible in domains where symmetry is a physical law:
- Molecular Design: In Generating Highly Designable Proteins with Geometric Algebra Flow Matching, Simon Wagner and colleagues use multivectors to capture the tetrahedral geometry of protein backbones. Similarly, Cong Liu uses Clifford diffusion in Clifford Group Equivariant Diffusion Models for 3D Molecular Generation to capture oriented planes and volumes during the generative process.
- High-Energy Physics: Particle physics happens in Minkowski spacetime, where Lorentz symmetry (rotations + velocity boosts) is paramount. Johann Brehmer and Víctor Bresó developed L-GATr, a transformer that natively understands the Minkowski metric. By representing energy and momentum as Clifford multivectors, the model achieves state-of-the-art results at the Large Hadron Collider (LHC).
- Knowledge Graphs: Even in non-physical domains, the expressive power of Clifford algebras helps. Chengjin Xu shows in Knowledge Graph Embeddings in Geometric Algebras that multivector embeddings can subsume complex and quaternion-based models, providing more degrees of freedom to capture nuanced relational patterns.
Why It Works: The Intuition
The reason Clifford-based models outperform standard ones is inductive bias. A standard network must learn that a rotated chair is still a chair by seeing thousands of examples. A Clifford Group Equivariant network knows the chair is the same because its very building blocks—the multivectors and the geometric product—are invariant to the choice of coordinates. By aligning the neural architecture with the mathematical structure of the physical world, these models stop fighting the geometry of the data and start using it.