Abstract
Neural networks often require large amounts of data to generalize and can be ill-suited for modeling small and noisy experimental datasets. Standard network architectures trained on scarce and noisy data will return predictions that violate the underlying physics. In this paper, we present methods for embedding even--odd symmetries and conservation laws in neural networks and propose novel extensions and use cases for physical constraint embedded neural networks. We design an even--odd decomposition architecture for disentangling a neural network parameterized function into its even and odd components and demonstrate that it can accurately infer symmetries without prior knowledge. We highlight the noise resilient properties of physical constraint embedded neural networks and demonstrate their utility as physics-informed noise regulators. Here we employed a conservation of energy constraint embedded network as a physics-informed noise regulator for a symbolic regression task. We showed that our approach returns a symbolic representation of the neural network parameterized function that aligns well with the underlying physics while outperforming a baseline symbolic regression approach.
Introduction
Standard neural networks are often viewed as "black boxes" that struggle to generalize when data is scarce or noisy. In scientific domains, experimental data is frequently expensive to collect and subject to measurement errors. When standard Multi-Layer Perceptrons (MLPs) are trained on such data, they often produce "unphysical" predictions—such as violating conservation laws or failing to respect inherent symmetries. This paper addresses the core challenge of Scientific Machine Learning (SciML): developing models that are data-driven yet physically consistent, interpretable, and resilient to noise.
Methods
The authors explore two primary strategies for embedding physical constraints directly into neural network architectures:
Even–Odd Symmetry Embedding
The research builds upon prior work using "Even-Odd Hub Neurons" in the final hidden layer to enforce specific symmetries. A novel contribution is the Even–Odd Decomposition Network. This architecture passes input through parallel "even" and "odd" sub-networks, allowing the model to discover the symmetry of a function without prior knowledge.
Conservation of Energy
To enforce conservation laws, the authors utilize Hamiltonian Neural Networks (HNNs). Instead of predicting state derivatives directly, an HNN learns to parameterize the Hamiltonian ($\mathcal{H}$), representing the system's total energy. By applying automatic differentiation to find the partial derivatives of $\mathcal{H}$, the network inherently respects energy conservation.
Physics-Informed Noise Regulation
A novel use case involves using an HNN as a noise regulator for symbolic regression (SINDy). SINDy typically fails with noisy data. By first training an HNN on noisy data and then using its "cleaned" predictions as input for SINDy, the authors can recover exact symbolic governing equations.
Results and Key Insights
The proposed methods were evaluated on synthetic datasets including simple functions ($x^2, x^3, e^x$) and dynamical systems (mass-spring, pendulum) with varying noise levels.
Decomposition and Symmetry Inference
The even-odd decomposition network successfully disentangled complex signals into their constituent parts. For example, it decomposed $e^x$ into its hyperbolic sine and cosine components without explicit supervision.
Noise Resilience and Symbolic Recovery
HNNs demonstrated superior noise resilience by being constrained to a specific energy manifold. While standard MLPs overfit to noise, HNNs maintained long-term stability in phase-space predictions. In symbolic regression tasks at noise levels of $\sigma=0.03$, the SINDy+HNN approach successfully recovered near-exact physical equations (e.g., $\dot{q} \approx 2p, \dot{p} \approx -2q$), whereas baseline SINDy produced overly complex and incorrect models.
Discussion
While effective, the current even-odd decomposition requires centered data to guarantee the simplest decomposition. Future work could incorporate complexity metrics directly into training to further improve symbolic fitting and explore scaling these constraints to high-dimensional, multi-body systems. By baking physical laws into the architecture, these models bridge the gap between messy experimental data and clean symbolic representations.
Summary updated