TL;DR
Current generative models for solving PDE inverse problems enforce physics as a hard constraint post-hoc, but this paper reveals they sample the wrong distribution due to a measure-theoretic oversight known as the Borel-Kolmogorov paradox. By introducing a co-area (Fixman) Jacobian correction, the authors present CoCoS, a sampler that restores true Bayesian posterior consistency and accurate uncertainty quantification.
What Problem Does This Paper Address?
When using diffusion or flow-matching models to solve partial differential equation (PDE) inverse problems, researchers often force the generated samples to exactly satisfy physical laws (like conservation of mass or boundary conditions). This is typically done post-hoc by projecting samples onto a valid "physics manifold" or using residual guidance.
However, because a hard constraint defines a zero-measure manifold in the state space, conditioning on it is mathematically ambiguous—a classic conundrum known as the Borel-Kolmogorov paradox. The physically meaningful approach assumes the PDE holds up to an infinitesimally small noise residual. Standard projection methods instead target a geometric "Euclidean limit," completely omitting a crucial co-area Jacobian factor (the Fixman correction). As a result, "satisfying the physics" does not mean you are correctly "sampling the posterior," leading to severely miscalibrated uncertainty estimates.
Methods
The authors propose CoCoS (Co-Area Constrained Sampling), an exact manifold MCMC sampler. Unlike standard projection methods that minimize Euclidean displacement, CoCoS explicitly incorporates the missing co-area weight into a Boltzmann potential:
$$V(x) = -\log \pi(x) - \log \ell(y|x) + \frac{1}{2}\log \det(J(x)J(x)^\top)$$
where $J$ is the constraint Jacobian, $\pi(x)$ is the prior, and $\ell(y|x)$ is the likelihood. It uses an isotropic tangent step proposal followed by a normal re-projection and a reverse-projection check to guarantee mathematical reversibility and exactness.
To bypass the high computational cost of test-time MCMC and Jacobian evaluations, they also introduce CoCo-Flow, an amortized version that distills the exact sampler into a conditional flow-matching model during training, allowing for instant inference.
Datasets
The methods were evaluated using a set of synthetic and physical PDE inverse problem benchmarks:
- A controlled 4D toy benchmark with a single non-linear quadratic constraint to isolate measure effects from PDE solver artifacts.
- A 1D Darcy Flow inverse problem (reconstructing log-permeability fields from sparse pressure observations) evaluated at dimensions $d=8$ and $d=16$.
- A 1D Steady Viscous Burgers inverse problem to test scenarios with mild sensitivity variation.
- A high-dimensional 2D Darcy Flow field-valued problem ($d=64$).
To evaluate accuracy without MCMC mixing bias, the authors implemented an i.i.d. rejection arbiter (residual-band rejection) to serve as the ground truth.
Key Insights and Ideas
- Geometry vs. Probability: Standard projection methods pile up samples where the constraint manifold is "flat" (low sensitivity) and deplete them where it is "steep" (high sensitivity). The co-area correction properly down-weights highly sensitive regions to match the physical residual-noise limit.
- Spurious Correlations: Omitting the Fixman factor doesn't just cause a minor shift; it fundamentally distorts the posterior covariance structure, creating entirely artificial correlations between variables that do not exist in the true posterior.
- Amortization Saves the Day: Forward-simulation-based training (generating data pairs via the forward model) naturally avoids this bias because it never explicitly conditions on a zero-measure manifold. CoCo-Flow leverages this to deliver calibrated posteriors in a single forward pass at test time.
Key Results
- Error Inflation: In the 4D benchmark, omitting the Fixman factor inflated the posterior error to 21$\times$ the sampling noise floor, while standard minimal-displacement projection (PCFM) sat at 9$\times$ the floor. CoCoS successfully matched the ground-truth floor.
- Covariance Distortion: On the 1D Darcy Flow problem ($d=8$), standard projection methods distorted the covariance matrix by 122% error and miscalculated the posterior standard deviation by 96%. CoCoS reduced covariance error to just 5%.
- Field-Scale Scaling: For the 2D Darcy Flow problem ($d=64$), the constraint sensitivity varied by over five orders of magnitude ($2.6 \times 10^5\times$). In this highly heterogeneous regime, CoCo-Flow successfully recovered the true posterior in the data-constrained subspace, while standard projection remained heavily biased.
Limitations and Future Work
- Computational Cost: The exact CoCoS sampler requires computing the constraint Jacobian $J$ and an $m \times m$ log-determinant at every step, which becomes expensive when the number of constraints ($m$) is very large.
- Scaling Amortization: While CoCo-Flow successfully shifts this computational burden to training time, fully scaling it to handle arbitrary observation operators and extremely high-dimensional field-valued parameters remains an open direction for future engineering optimization.
Summary updated