Abstract
Solving inverse problems—reconstructing a complete "true" state from partial or noisy observations—is a fundamental challenge in scientific computing. Traditional approaches rely on minimizing hand-crafted variational cost functions, which often fail to capture complex data distributions and suffer from slow optimization. We propose an end-to-end supervised framework that jointly learns a variational representation (the "energy landscape") and a neural-network-based solver (the "pathfinder"). By training these components together, we achieve significantly better and faster reconstructions for image inpainting and chaotic physical systems compared to traditional methods.
Introduction
Inverse problems are typically ill-posed, meaning the observed data $y$ does not uniquely determine the state $x$. To resolve this, variational regularization is used to define a cost function where the global minimum represents the most likely solution. However, this approach faces two major hurdles: the model gap, where hand-designed regularizers fail to capture complex textures, and the solver gap, where standard optimization algorithms like gradient descent are slow or get trapped in local minima.
This work addresses these gaps by treating the entire inversion process as a meta-learning problem—learning how to optimize.
Methodology
The proposed framework consists of two primary components:
- Variational Representation ($\\Phi$): A neural network (e.g., CNN or Autoencoder) parameterizes the regularization term. The cost function penalizes the distance between the state and its projection by this network.
- Learned Solver ($\\Gamma$): A Convolutional LSTM performs iterative updates. Instead of standard gradient descent, the LSTM takes the gradient of the variational cost as input and learns the optimal update step at each iteration.
The system is trained end-to-end using pairs of ground-truth states and partial observations. This joint training allows the model to optimize the "shape" of the cost function specifically to facilitate the solver's strategy.
Experimental Results
The framework was evaluated on MNIST digit inpainting and chaotic physical systems (Lorenz-63 and Lorenz-96).
MNIST Inpainting
For the MNIST task, digits were corrupted with noise and three 9x9 occlusions. The jointly learned CNN-based model (2S-CNN + LSTM-S) achieved an I-score (error in missing areas) of 0.33, significantly outperforming PCA-based methods (I-score of 1.09). Visual results demonstrate that the joint approach produces much clearer and more topologically accurate reconstructions.
Chaotic Physical Systems
In experiments with Lorenz-96 models, the framework achieved a reconstruction error (R-score) of $1.94 \\times 10^{-2}$, nearly three times better than gradient descent performed on the true physical model ($5.65 \\times 10^{-2}$). This suggests that data-driven regularizers can effectively compensate for numerical instabilities or local minima inherent in physical equations.
Discussion and Conclusion
A key finding is that the best representation for data compression is not necessarily the best for inversion. Joint learning allows the model to sacrifice some representational accuracy to create a cost landscape that is easier for the solver to navigate. While the LSTM solver currently acts as a "black box," it typically converges in 10-20 steps, compared to hundreds required by classical methods. Future work will focus on improving the interpretability and convergence guarantees of these learned solvers.
Summary updated