GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras

Ekaterina Filimoshina, Dmitry Shirokov

Abstract

Equivariant neural networks (ENNs) are essential for modeling physical systems with inherent symmetries, such as rotations and reflections. While Clifford Geometric Algebra (GA) provides a robust framework for constructing these networks, current state-of-the-art models suffer from extreme overparameterization and a tendency to overfit. This paper presents GLGENN (Generalized Lipschitz Group Equivariant Neural Networks), an architecture that leverages generalized Lipschitz groups and a novel weight-sharing scheme based on the four fundamental subspaces of geometric algebra. GLGENN achieves state-of-the-art performance across physics and geometry benchmarks while utilizing less than half the parameters of existing GA-based models, demonstrating superior generalization and computational efficiency.

Introduction

In the natural sciences, physical laws are often invariant or equivariant under specific transformations. For example, the gravitational interaction between particles remains the same regardless of the coordinate system's orientation. Equivariant Neural Networks (ENNs) bake these symmetries—rotation, translation, and reflection—directly into the model architecture, allowing for better generalization from smaller datasets.

Recently, Geometric Algebra (GA) has emerged as a powerful mathematical language for ENNs. Models like Clifford Group Equivariant Neural Networks (CGENN) have set new benchmarks by representing geometric objects (scalars, vectors, bivectors) within a unified framework. However, these models face a significant hurdle: overparameterization. By assigning independent learnable parameters to every geometric "grade," these networks become computationally expensive and prone to overfitting, particularly in scientific domains where data is scarce. This paper introduces GLGENN, a parameter-light alternative that maintains high expressivity without the associated bloat.

Methodology

Generalized Lipschitz Groups

The core innovation of GLGENN is the introduction of a new mathematical class of Lie groups known as generalized Lipschitz groups. The authors provide a theoretical proof that a neural network equivariant to these broader groups is inherently equivariant to standard pseudo-orthogonal groups (the rotations and reflections relevant to physics).

Counterintuitively, designing for a broader symmetry group reduces the number of free parameters. Because the mathematical requirements for equivariance under a larger group are more stringent, the degrees of freedom available for the network's weights are naturally constrained. This leads to a more "economic" and efficient parameterization.

Subspace Weight-Sharing

To implement this efficiency, GLGENN utilizes a novel weight-sharing scheme. Geometric algebras can be decomposed into four fundamental subspaces defined by two operations: grade involution and reversion. While previous models like CGENN treated each geometric grade as a separate entity requiring its own weights, GLGENN processes geometric objects in unified groups of four. By projecting inputs onto these four subspaces, the model shares weights across different grades. The authors developed custom linear, geometric product, and normalization layers specifically designed to respect this 4-subspace structure.

Experimental Evaluation

The authors evaluated GLGENN against state-of-the-art baselines on several synthetic and physics-based benchmarks:

  1. O(5,0)-Regression Task: A synthetic task requiring the model to estimate a specific mathematical function of vectors sampled from a Gaussian distribution.
  2. Convex Hull Volume Estimation: Predicting the volume of a convex hull generated by $K$ points in 5D and 7D space (with $K$ ranging from 16 to 512).
  3. O(5,0)-N-Body Experiment: A simulation of five interacting charged particles evolving under Newtonian gravity, testing the model's ability to predict final positions based on initial states.

Results and Discussion

Parameter Efficiency and Training Speed

GLGENN consistently utilized roughly half the parameters of CGENN. In the Convex Hull task ($K=16$), GLGENN required only 24.1K parameters compared to CGENN’s 58.8K. Similarly, in the N-Body task, GLGENN used 103K parameters versus 210K for CGENN. This reduction in complexity translated directly into faster training times, with GLGENN requiring significantly less wall-clock time per training batch.

Generalization and Overfitting

Despite its smaller size, GLGENN matched or exceeded the performance of larger models. On small datasets (e.g., 256 or 1024 samples), CGENN exhibited classic signs of overfitting: its training loss dropped to zero while test loss plateaued. In contrast, GLGENN maintained a tight alignment between training and test loss, demonstrating superior generalization. Furthermore, when CGENN was artificially constrained to match GLGENN’s parameter count, its performance degraded significantly, proving that GLGENN’s architecture is fundamentally more efficient rather than just smaller.

Conclusion

GLGENN addresses the critical issue of overparameterization in GA-based equivariant networks. By leveraging the mathematical properties of generalized Lipschitz groups and fundamental subspaces, the architecture provides a robust, efficient, and high-performing solution for scientific machine learning. While the current study focuses on non-degenerate geometric algebras and simulated datasets, future work will explore degenerate algebras and the application of GLGENN to real-world data, such as protein folding and fluid dynamics.

Summary updated

BibTeX
@inproceedings{filimoshina2025glgenn,
  author = {Ekaterina Filimoshina and Dmitry Shirokov},
  title = {{GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=H0ySAzwu8k},
}
Figure