Temporal Causality in Physics-Informed Neural Networks

This direction is split into other directions: temporal causality violation. Its last version stays readable and citable here.

Version 1 of 149 papers58 conceptsfrozen 2026-10-04

Standard Physics-Informed Neural Networks (PINNs) treat time as just another spatial dimension. When solving a partial differential equation (PDE), a PINN typically takes a coordinate $(x, t)$ and predicts a value $u(x, t)$, using the PDE residual as a loss function. While elegant, this approach ignores a fundamental physical reality: the arrow of time. In the physical world, the state of a system at $t=1$ is strictly caused by its state at $t=0.9$. By treating all time points as independent and simultaneous, standard PINNs often fail to converge or produce non-physical solutions.

The Failure of Simultaneous Optimization

In traditional numerical solvers, we "march" through time. We solve for the next step based on the current one. In contrast, Maziar Raissi and Nazanin Ahmadi Daryakenari note in Physics-Informed Neural Networks and Extensions that PINNs attempt to solve the entire space-time domain at once.

This "global-in-time" optimization creates a massive, high-dimensional landscape where the optimizer can easily get lost. As Sifan Wang explains in Respecting causality for training physics-informed neural networks, the network might find a "shortcut" that minimizes the total residual by fitting future time steps before it has accurately resolved the initial conditions. This violation of causality is particularly catastrophic for chaotic or turbulent systems, where small errors at the start lead to massive divergences later.

Causal Weighting and Progressive Learning

To fix this, researchers have introduced mechanisms that force the network to respect temporal order. The intuition is simple: the network should not be allowed to learn what happens at $t=10$ until it has mastered $t=1$.

Sifan Wang and Paris Perdikaris proposed a causal weighting strategy. In An Expert's Guide to Training Physics-informed Neural Networks, they describe a loss function where the residuals at later time steps are weighted by the network's performance on earlier ones. If the error at $t=0.1$ is high, the weights for all $t > 0.1$ are pushed toward zero. This effectively "unfolds" the optimization problem, forcing the model to learn the physics sequentially.

Similarly, Jia Guo and Haifeng Wang developed Adaptive Causal Sampling in A Novel Adaptive Causal Sampling Method for Physics-Informed Neural Networks. Instead of weighting the loss, they change where the model looks, dynamically focusing training points on the "frontier" where the solution is currently being resolved.

Breaking Propagation Barriers

A major hurdle in PINNs is "propagation failure"—the inability of information from the initial boundary to reach the interior of the domain. Arka Daw and Anuj Karpatne address this in Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling.

Their R3 Sampling algorithm acts like a filter. It retains points where the model is struggling (high residual) and releases points where the physics have been mastered. By coupling this with temporal causality, they ensure that the "mastery" of the solution propagates forward from $t=0$ like a wave, preventing the model from converging to trivial, non-physical states.

Architectural Inductive Biases

Beyond changing the loss function, a new wave of research suggests that the standard Multi-Layer Perceptron (MLP) is fundamentally the wrong tool for time-dependent physics. These researchers argue for architectures with a built-in "recursive" or "marching" bias.

Why Causality Works

The shift toward causal training works because it aligns the optimization process with the mathematical structure of the underlying equations. Evolution equations are, by definition, sequential. When we force a neural network to respect this sequence, we transform a nearly impossible global search into a series of manageable, local updates. This not only improves accuracy by several orders of magnitude but also enables PINNs to tackle the chaotic, high-frequency dynamics that were previously out of reach.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_causality_in_physics_informed_n_4a34a8c2,
  title = {Temporal Causality in Physics-Informed Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-causality-in-physics-informed-neural-networks/rdr_4a34a8c2c1674927b5c3335f31c1dbc3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_temporal_causality_in_physics_informed_n_4a34a8c2_v1,
  title = {Temporal Causality in Physics-Informed Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-causality-in-physics-informed-neural-networks/rdr_4a34a8c2c1674927b5c3335f31c1dbc3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}