Automated Symmetry Discovery via Infinitesimal Generators

32 concepts24 papers

Symmetry is one of the most powerful inductive biases in machine learning. If a model knows that an image's meaning is invariant to translation, or that a physical system's dynamics are invariant to rotation, it can eliminate vast regions of the search space, learning faster and generalizing better with less data.

Traditionally, engineers have had to hard-code these symmetries into neural network architectures—such as using translations in Convolutional Neural Networks (CNNs) or permutations in Graph Neural Networks. But what happens when we encounter complex, real-world datasets where the underlying symmetries are unknown, non-linear, or hidden behind high-dimensional observations?

A rapidly growing body of research addresses this challenge through automated symmetry discovery. Instead of forcing human experts to mathematically derive and hard-code symmetries, these techniques allow neural networks to discover continuous symmetry groups directly from raw data.

The core insight that makes this possible is a shift from global transformations to infinitesimal generators. By optimizing local, linear tangent spaces (Lie algebras) rather than searching over complex global transformation groups (Lie groups), symmetry discovery becomes a stable, differentiable optimization problem.


The Core Insight: Thinking Infinitesimally

Searching for an unknown global symmetry group is notoriously difficult. Continuous transformations—like rotating an image by any arbitrary angle or applying a Lorentz boost to particle trajectories—are non-linear and form complex geometric manifolds. Directly optimizing over the space of all possible global transformations is computationally intractable and highly unstable.

To make this problem differentiable, researchers leverage Lie theory. A continuous transformation group (a Lie group) can be completely characterized by its behavior infinitely close to the identity transformation. The tangent space at this identity is a vector space known as a Lie algebra, and its basis elements are called infinitesimal generators.

Global Lie Group (e.g., Full Rotations)
        ▲
        │  Exponential Map: exp(θ · A)
        │
Infinitesimal Generator / Lie Algebra Basis (Matrix A)

Think of a full rotation around a sphere. Instead of trying to learn the entire non-linear rotation matrix directly, a network can learn a single vector field that points in the direction of the rotation at the north pole. Because the Lie algebra is a linear vector space, we can parameterize potential symmetries as a linear combination of learnable basis matrices or vector fields.

To recover the global transformation from these local tangents, we use the matrix exponential ($\exp(\theta A)$) or integrate the vector field. This mathematical bridge ensures that whatever local structure the network learns, the resulting global operations are guaranteed to satisfy the strict algebraic properties of a group (such as containing inverses and maintaining closure).


How Networks Parameterize and Learn the Basis

To automatically discover symmetries, architectures must represent these infinitesimal generators in a way that gradient descent can optimize.

Learnable Basis Matrices

As shown by Dehmamy et al. in their Lie Algebra Convolutional Network (L-conv), the basis elements of a Lie algebra can be treated as learnable parameters. Instead of hard-coding a specific symmetry group, the network learns a set of matrices that span the symmetry space of the data. By exponentiating these learned matrices, the network automatically constructs equivariant convolutional filters tailored to the dataset's unique geometry.

Similarly, Gabel et al. demonstrate that neural networks can autonomously converge to the exact mathematical coefficients of continuous symmetries by learning linear differential operators that represent these infinitesimal transformations. To expand the expressivity beyond simple linear transformations, Gabel et al. parameterize the infinitesimal generators using a basis of quadratic functions, allowing the system to capture a wider variety of non-affine geometric shifts.

Vector Field Parameterization

When symmetries are highly non-linear, representing them as fixed matrices fails. To overcome this, Ko et al. model continuous symmetries as learnable vector fields that act as the infinitesimal generators of transformations. This allows the model to discover complex, non-linear symmetries directly from data without pre-defining any geometric groups.


Optimization Objectives: How to Find a Symmetry

How does a model know it has found a valid symmetry? Current research relies on three primary optimization paradigms: gradient annihilation, generative alignment, and flow matching.

1. Gradient Annihilation and Invariance

If a target function $f(x)$ (such as a classifier or a physical law) is invariant to a continuous transformation, then moving an input $x$ along the symmetry's tangent vector field should not change the function's output. In other words, the directional derivative of the function along the generator must be zero.

Shaw et al. frame symmetry discovery as finding tangent vector fields that "annihilate" machine learning functions. This transforms a global search into a stable, local optimization problem that avoids discrete sampling.

Alternatively, we can inspect already-trained models to see what symmetries they have implicitly internalized. Moskalev et al. introduced LieGG, a framework that extracts infinitesimal generators directly from a trained model by constructing a polarization matrix from the network's input gradients. By identifying the nullspace of this matrix via Singular Value Decomposition (SVD), they mathematically expose the continuous symmetries the network naturally discovered during training.

2. Generative Alignment (LieGAN)

In generative contexts, we want to discover the symmetries inherent in a data distribution. Yang et al. developed LieGAN, which uses an adversarial framework. A generator learns a Lie algebra basis to transform real data points, while a discriminator ensures that the transformed data remains indistinguishable from the original dataset. This forces the learned basis to align perfectly with the true underlying symmetries of the data distribution.

To handle highly non-linear group actions, Yang et al. extended this concept to Latent LieGAN (LaLiGAN). LaLiGAN maps high-dimensional observations into a latent space where complex, non-linear symmetries become linear, allowing the model to approximate smooth, proper group actions via simple latent matrix operations.

3. Flow Matching

While adversarial training can be unstable, generative symmetry discovery can also be framed as density estimation over a group manifold. Chen et al. introduced LieFlow, which applies flow matching to model distributions over Lie groups. By using a vector field to transform a simple prior into a complex distribution on the group manifold, they unify the discovery of both continuous and discrete symmetries within a single, stable generative framework.


Why It Works: Applications and Impact

By shifting the burden of symmetry derivation from humans to algorithms, infinitesimal generator learning unlocks new capabilities across several domains.

Unsupervised Disentanglement

In representation learning, a major goal is to separate independent factors of variation (e.g., separating an object's identity from its orientation). Chau et al. developed Lie Group Sparse Coding (LSC), demonstrating that embedding the mathematical structure of a transformation group allows a model to extrapolate geometric transformations far beyond the narrow range seen during training.

Similarly, Keurti showed that learning the underlying generators of a transformation group enables a Homomorphism Autoencoder (HAE) to perform full-range traversals (like a complete $360^\circ$ rotation) even when exposed only to tiny, incremental changes during training. To organize these factors without supervision, Kim introduced a learnable symmetry codebook that optimizes distinct sections of a Lie algebra to represent independent factors of variation.

Raw Transitions ──► Homomorphism Autoencoder ──► Disentangled Latent Space
                       (Learns Lie Algebra)          (Rotation, Scale, Translation)

Discovering Physical Laws

In scientific computing, knowing the conservation laws of a dynamical system is often the key to predicting its long-term behavior. Lishkova et al. combined variational integrators with Lie group generators in their SymDLNN framework, allowing a model to automatically discover hidden physical constants (like angular momentum) directly from raw trajectory data.

When the governing equations of a system are completely unknown, Yang et al. demonstrated that learning infinitesimal generators from raw data can be used to constrain symbolic regression. By restricting the search space exclusively to equations that obey the discovered symmetries, the accuracy and noise-resilience of automated scientific discovery are dramatically improved.

Ultimately, as validated by frameworks like AtlasD (Bhat et al.), data-driven symmetry identification is reaching a level of maturity where automatically discovered local and global symmetry groups produce models that rival the performance of architectures manually engineered by domain experts. This transition from hard-coded geometry to differentiable geometric discovery marks a profound shift toward truly self-organizing artificial intelligence.

Key Foundational Work

Automatic Symmetry Discovery with Lie Algebra Convolutional Network (2021) — contributed 2 concept elements to this direction.
Deep Autoencoders: From Understanding to Generalization Guarantees (2021) — contributed 2 concept elements to this direction.
LieGG: Studying Learned Lie Group Generators (2022) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_infinitesimal_generator_learning_acc0ebc2,
  title = {infinitesimal generator learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/infinitesimal-generator-learning/rdr_acc0ebc2da2d44d286ad8e730bd98eaf},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_infinitesimal_generator_learning_acc0ebc2_v1,
  title = {infinitesimal generator learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/infinitesimal-generator-learning/rdr_acc0ebc2da2d44d286ad8e730bd98eaf/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake