In the rapidly growing field of physics-informed machine learning (PIML), the standard recipe is simple: take a neural network and sprinkle in some physical laws—usually as a penalty in the loss function or a specialized layer. But does this actually make the model better, or are we just adding complexity for the sake of "physicality"?
A new paper, "Where Should Physics Enter? Information, Identifiability and Calibration in Probabilistic Sea Surface Temperature Forecasting," provides a rigorous reality check. By dissecting how advection-diffusion priors affect sea surface temperature (SST) forecasts, the authors reveal that many common PIML practices might be providing a placebo effect rather than a physical edge.
TL;DR
For sea surface temperature forecasting, common "physics-informed" tricks like soft residuals or latent transport layers often fail to provide measurable benefits on real-world data. Instead, the most significant performance gains come from treating physical variables (like ocean currents) as raw input information and applying simple post-hoc calibration to fix lead-dependent uncertainty.
What Problem Does This Paper Address?
Predicting sea surface temperature is vital for understanding everything from monsoon patterns to cyclone intensity. While deep learning models like ConvLSTMs are efficient at this task, researchers often try to make them "physics-informed" by incorporating advection-diffusion equations.
The authors set out to answer three critical questions:
- Does the physics actually help? Does adding a physical prior improve forecast accuracy on real data?
- Is the physics "identifiable"? When a model learns a "latent velocity" to move heat around, is it actually learning real ocean currents, or just a mathematical fudge factor?
- Is "physics" just a proxy for something else? Could the perceived benefits of PIML actually be coming from better data (information) or better-tuned uncertainty (calibration)?
Notably, this paper serves as a formal withdrawal of an earlier claim by the same group that physics-informed methods reduced error by 81%—a result they now attribute to an uncontrolled comparison.
Methods
The researchers used a heteroscedastic Gated-Attention ConvLSTM as their backbone. "Heteroscedastic" means the model predicts both the expected temperature (mean) and the uncertainty (variance) at every pixel.
They tested three ways to "enter" physics into the model:
- As a Regularizer: Adding a soft advection-diffusion residual to the loss function.
- As a Structure: Including an explicit "transport step" (a semi-Lagrangian warp) inside the neural network architecture.
- As Information: Simply feeding external ocean current data (velocities) into the network as additional input channels.
To validate their findings, they developed a synthetic benchmark on the same grid as the real-world data, where the "ground truth" physics and currents were perfectly known and controlled.
Datasets
- Real-World Data: Daily NOAA OISST v2.1 data for the Bay of Bengal (1982–2024).
- Ocean Currents: GLORYS12 reanalysis currents and OSCAR surface currents were used for both model input and evaluation of learned velocities.
- Synthetic Benchmark: A custom-generated dataset using divergence-free eddies to simulate heat transport, allowing for perfectly controlled experiments.
Key Insights and Ideas
- The Identifiability Crisis: Perhaps the most striking finding is that "learned velocities" are essentially junk. In every model tested, the velocity fields the network came up with correlated with real ocean currents no better than random noise. The model has an "ambiguity" problem: it can achieve the same result by either moving heat physically or just "painting" the correction onto the map.
- Information > Structure: In synthetic tests where currents actually mattered, simply giving the network the velocity data as an input channel captured 92% of the benefit of a complex, explicit physical transport layer. The "physics" wasn't helping because of its mathematical structure, but because it was useful data.
- Real-World Mismatch: On real data, using high-quality reanalysis currents (GLORYS) actually made forecasts worse at short leads. The reason? The advection described by these currents explained almost none of the day-to-day temperature changes in the OISST dataset, which are likely dominated by surface heat fluxes and mixing that the simple advection model ignores.
Key Results
- Null Result on Real Data: On 43 years of Bay of Bengal data, the soft physics residual provided no reliable benefit over a simple "temporal smoothness" penalty.
- Calibration is the Real Hero: Every model tested suffered from a shared defect: they were under-confident at short leads and over-confident at long leads. By applying a simple 15-parameter post-hoc variance correction (one multiplier per forecast lead), the authors improved the Continuous Ranked Probability Score (CRPS) by 13–21%—far more than any "physics" modification.
Limitations and Future Work
The authors are careful to note that their "null result" applies specifically to the Bay of Bengal at a 0.25° resolution. Physics might still be the key in other contexts:
- High-Resolution Data: At finer scales, where temperature gradients are sharper, advection might play a more dominant role.
- Different Regions: Areas with strong, consistent currents (like the Gulf Stream) might show more benefit from transport priors.
- Supervised Velocity: Instead of letting the model "guess" a latent velocity, future work should explore supervising the velocity head with real data or using data assimilation techniques.
The paper concludes with a piece of advice for the PIML community: before claiming a physical prior works, always compare it against a "no-physics" baseline that receives the same information, and always check if your "learned" physical variables actually resemble reality.
Summary updated