Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning

Haoran Li, Chenhan Xiao, Muhao Guo, Yang Weng

Abstract

This paper introduces Latent Mixture of Symmetries (Latent MoS), a framework designed to learn complex system dynamics from sparse, low-resolution data. By leveraging a "Mixture of Experts" grounded in Lie group theory, Latent MoS adaptively combines different latent geometric transformations—such as rotation, translation, and scaling—to capture real-world "broken" symmetries. Our evaluation across eight diverse physical and engineering datasets demonstrates that Latent MoS significantly improves sample efficiency, interpretability, and predictive accuracy compared to existing state-of-the-art models.

Introduction: The Challenge of Real-World Dynamics

In fields like robotics and power systems, learning the underlying "rules" of a dynamic system from sensor data is essential. However, real-world data is frequently sparse and irregular due to sensor limitations or communication failures. While researchers often use symmetry (equivariance) as a prior to help models learn faster, most existing methods assume a single, global symmetry. Real physical systems often exhibit symmetry breaking, where symmetries hold only locally or involve multiple overlapping factors—like a pendulum that is both swinging (rotation) and slowing down (scaling/damping). Current models struggle to represent these "mixed" dynamics, often leading to overfitting when data is scarce.

Methods: Embedding Geometric Priors

Latent MoS addresses these challenges by embedding geometric priors directly into a latent space. The architecture consists of four main components:

  1. Latent ODE Foundation: It uses a Latent Ordinary Differential Equation (ODE) framework with an ODE-RNN encoder to handle irregularly sampled and sparse time series naturally.
  2. Mixture of Symmetry Experts (MoE): Instead of a generic neural network, the model uses a gated mixture of Lie group experts. Each expert is restricted to a specific type of geometric transformation: Planar Rotation ($SO(2)$), Translation ($E(m)$), Scaling ($Sim(m)$), and second-order compositions (e.g., rotation + scaling).
  3. Gating Mechanism: A neural network acts as a "conductor," learning to assign weights to these experts based on the current state and time, allowing the model to switch between different physical regimes dynamically.
  4. Hierarchical Stacking: To capture both quick, local changes and long-term trends, the model stacks multiple MoS blocks operating at different temporal resolutions.
The Latent MoS architecture showing the integration of Lie group experts.

Key Insights: Symmetry as a Physical Constraint

By forcing latent transitions to be linear combinations of Lie group actions, the model is physically "regularized." It cannot learn transitions that do not correspond to valid geometric transformations, which prevents it from "hallucinating" noise as dynamics. The "Mixture" approach allows the model to approximate complex, non-linear "symmetry-breaking" behaviors while maintaining mathematical rigor. Furthermore, unlike standard Neural ODEs that produce erratic latent trajectories, Latent MoS produces clean, geometric paths, making its internal representations useful for downstream safety-critical analysis.

Key Results

Latent MoS was evaluated across eight datasets, including synthetic ODE systems, energy systems, and biomedical signals. Key findings include:

  • Superior Sample Efficiency: In scenarios with a 90% data drop rate, Latent MoS consistently outperformed state-of-the-art baselines like Contiformer and standard Latent ODEs.
  • Massive Error Reduction: Across all datasets, the model achieved relative improvements in Mean Squared Error (MSE) ranging from 15% to 97% for both interpolation and extrapolation tasks.
  • Visual Fidelity: Qualitative results demonstrate that Latent MoS reconstructs the underlying system geometry far more accurately than standard Latent ODEs.
Qualitative comparison of latent trajectories across different systems.
  • Expert Identification: In the Glycolytic system, the gating mechanism correctly identified a "rotation × scaling" expert as the dominant factor (97% weight), proving the model learns the underlying physics.

Limitations and Future Work

The model currently relies on a predefined set of Lie group experts. If a system possesses a highly exotic symmetry not covered by these, the model's performance may be limited. Additionally, calculating matrix exponentials for many experts can be computationally intensive. Future work will focus on automated ways to discover and refine the set of candidate symmetries and applying these structured latent spaces to more complex stability and safety-critical control problems.

Summary updated

BibTeX
@inproceedings{li2025latent,
  author = {Haoran Li and Chenhan Xiao and Muhao Guo and Yang Weng},
  title = {{Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=t7NLyOtPEi},
}
Figure