Physics Residuals as Unsupervised Error Signals

17 concepts8 papers1 proposal

In traditional machine learning, determining whether a model's prediction is accurate requires a ground-truth label. In scientific computing—where neural networks are trained to emulate complex physical systems governed by partial differential equations (PDEs)—generating these labels requires running computationally expensive numerical solvers. This creates a paradox: we use machine learning to avoid expensive simulations, but we need those very simulations to discover where our models fail.

However, scientific machine learning possesses a unique advantage. Even when we do not know the true solution for a given input, we know the exact mathematical laws the solution must obey. By plugging a model's prediction directly into the governing differential equations or conservation laws, we can compute a physics residual. If the residual is zero, the prediction perfectly satisfies the physical laws; if it is high, the model has generated a physically impossible state.

This insight transforms the physics residual from a mere loss function component into a powerful, unsupervised proxy for model error and uncertainty. By treating the residual as an inherent error signal, recent work demonstrates how we can perform active learning, select optimal training subsets, and guarantee uncertainty bounds without needing ground-truth simulation data.


The Intuition: Why Residuals Map to Error

To understand why this works, imagine a neural operator predicting fluid dynamics over time. Traditional uncertainty estimation techniques (like deep ensembles or MC Dropout) measure variance—how much different model initializations or architectures disagree with one another. However, a model can be highly confident yet completely wrong, agreeing on a prediction that violates the conservation of mass.

Evaluating the physics residual bypasses this issue entirely. As demonstrated by Alicja Polanska et al. in Data-Efficient Neural Operator Training via Physics-Based Active Learning, evaluating the differential operator on a predicted solution trajectory yields a direct, physically grounded indicator of inaccuracy. Because the residual requires no ground-truth labels, it can be computed instantly anywhere in the domain or parameter space. This provides a direct window into the model's internal physical consistency, serving as an intuitive and zero-cost metric for guiding data selection.


Smart Data Acquisition via Active Learning

Because high residuals pinpoint exactly where a model's internal representation of physics is weakest, they serve as an ideal acquisition function for active learning. Instead of uniformly sampling a parameter space or randomly selecting scenarios to simulate, a "greedy" strategy can be deployed: evaluate the model across a vast unlabeled pool, compute the residuals, and only run expensive numerical simulations for the samples with the highest physics violations.

Several frameworks leverage this loop to drastically reduce data requirements:

Remarkably, this selection can happen before any training begins. Anant Khandelwal et al. developed PICore in PICore: Physics-Informed Unsupervised Coreset Selection for Data Efficient Neural Operator Training, a framework that uses physics-informed losses as an unsupervised coreset selection tool. By identifying high-residual inputs from an unlabeled pool upfront, they isolate the most informative training samples before running any expensive simulations, boosting end-to-end training efficiency by up to 78%.


Rigorous Uncertainty Quantification and Filtering

Beyond guiding data collection, physics residuals offer a robust mechanism for quantifying uncertainty and filtering model outputs during inference.

A major challenge in scientific machine learning is guaranteeing that a model's confidence intervals are statistically valid. To address this, Vignesh Gopakumar et al. in Calibrated Physics-Informed Uncertainty Quantification utilize PDE residuals as a nonconformity score within a Conformal Prediction framework. Because the residual measures deviation from governing physics rather than ground-truth labels, this data-free approach provides statistically guaranteed uncertainty bounds, successfully catching "confident but wrong" predictions without requiring validation simulations.

[ Unlabeled Input ] ──> [ Neural Solver ] ──> [ Prediction ]
                                                    │
                                                    ▼
                                        [ Evaluate PDE Operators ]
                                                    │
                                                    ▼
                                        [ Physics Residual Error ]
                                         /                      \
                                        ▼                        ▼
                        [ Active Learning Query ]      [ Conformal Prediction Bounds ]
                        (If residual is high,          (Maps residual to statistical
                         trigger simulation)            uncertainty guarantee)

Similarly, residuals can be integrated directly into generative architectures to enforce physical validity. In the PID-GAN framework introduced by Arka Daw et al. in PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics, a physics consistency score is mapped from the residual and fed as an auxiliary input to the discriminator, forcing the generator to respect physical legality.

Alternatively, when dealing with probabilistic models where constraints are not explicitly enforced during training, residuals can act as a post-inference filter. Katsiaryna Haitsiukevich et al. in Diffusion models as probabilistic neural operators for recovering unobserved states of dynamical systems use diffusion models to generate multiple candidate states for dynamical systems. By calculating the PDE residual error for each candidate post-inference, they can reliably select the most physically accurate prediction from the generated distribution without requiring physics-informed constraints during training.

Key Foundational Work

Active Learning with Physics-Informed Graph Neural Networks on Unstructured Meshes (2024) — contributed 2 concept elements to this direction.
Diffusion models as probabilistic neural operators for recovering unobserved states of dynamical systems (2024) — contributed 1 concept elements to this direction.
PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_residual_driven_active_learning_7fd423c7,
  title = {residual-driven active learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/residual-driven-active-learning/rdr_7fd423c7266247ad96b456368ce78724},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_residual_driven_active_learning_7fd423c7_v1,
  title = {residual-driven active learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/residual-driven-active-learning/rdr_7fd423c7266247ad96b456368ce78724/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake