Joint learning of variational representations and solvers for inverse problems with partially-observed data

Ronan Fablet, Lucas Drumetz, François RousseauarXiv 2020 · arXiv

Abstract

Solving inverse problems—reconstructing a complete "true" state from partial or noisy observations—is a fundamental challenge in scientific computing. Traditional approaches rely on minimizing hand-crafted variational cost functions, which often fail to capture complex data distributions and suffer from slow optimization. We propose an end-to-end supervised framework that jointly learns a variational representation (the "energy landscape") and a neural-network-based solver (the "pathfinder"). By training these components together, we achieve significantly better and faster reconstructions for image inpainting and chaotic physical systems compared to traditional methods.

Introduction

Inverse problems are typically ill-posed, meaning the observed data $y$ does not uniquely determine the state $x$. To resolve this, variational regularization is used to define a cost function where the global minimum represents the most likely solution. However, this approach faces two major hurdles: the model gap, where hand-designed regularizers fail to capture complex textures, and the solver gap, where standard optimization algorithms like gradient descent are slow or get trapped in local minima.

This work addresses these gaps by treating the entire inversion process as a meta-learning problem—learning how to optimize.

Methodology

The proposed framework consists of two primary components:

  1. Variational Representation ($\\Phi$): A neural network (e.g., CNN or Autoencoder) parameterizes the regularization term. The cost function penalizes the distance between the state and its projection by this network.
  2. Learned Solver ($\\Gamma$): A Convolutional LSTM performs iterative updates. Instead of standard gradient descent, the LSTM takes the gradient of the variational cost as input and learns the optimal update step at each iteration.

The system is trained end-to-end using pairs of ground-truth states and partial observations. This joint training allows the model to optimize the "shape" of the cost function specifically to facilitate the solver's strategy.

Figure 1: Comparison of optimization trajectories. The learned solver (solid lines) finds a more direct and accurate path to the true state than standard gradient descent (dashed lines) on the same cost landscape.

Experimental Results

The framework was evaluated on MNIST digit inpainting and chaotic physical systems (Lorenz-63 and Lorenz-96).

MNIST Inpainting

For the MNIST task, digits were corrupted with noise and three 9x9 occlusions. The jointly learned CNN-based model (2S-CNN + LSTM-S) achieved an I-score (error in missing areas) of 0.33, significantly outperforming PCA-based methods (I-score of 1.09). Visual results demonstrate that the joint approach produces much clearer and more topologically accurate reconstructions.

Figure 2: Visual comparison of MNIST inpainting results across different methods. The bottom row shows the superior reconstruction quality of the jointly learned 2S-CNN-based operator compared to PCA and Autoencoder baselines.

Chaotic Physical Systems

In experiments with Lorenz-96 models, the framework achieved a reconstruction error (R-score) of $1.94 \\times 10^{-2}$, nearly three times better than gradient descent performed on the true physical model ($5.65 \\times 10^{-2}$). This suggests that data-driven regularizers can effectively compensate for numerical instabilities or local minima inherent in physical equations.

Discussion and Conclusion

A key finding is that the best representation for data compression is not necessarily the best for inversion. Joint learning allows the model to sacrifice some representational accuracy to create a cost landscape that is easier for the solver to navigate. While the LSTM solver currently acts as a "black box," it typically converges in 10-20 steps, compared to hundreds required by classical methods. Future work will focus on improving the interpretability and convergence guarantees of these learned solvers.

Summary updated

Abstract

Designing appropriate variational regularization schemes is a crucial part of solving inverse problems, making them better-posed and guaranteeing that the solution of the associated optimization problem satisfies desirable properties. Recently, learning-based strategies have appeared to be very efficient for solving inverse problems, by learning direct inversion schemes or plug-and-play regularizers from available pairs of true states and observations. In this paper, we go a step further and design an end-to-end framework allowing to learn actual variational frameworks for inverse problems in such a supervised setting. The variational cost and the gradient-based solver are both stated as neural networks using automatic differentiation for the latter. We can jointly learn both components to minimize the data reconstruction error on the true states. This leads to a data-driven discovery of variational models. We consider an application to inverse problems with incomplete datasets (image inpainting and multivariate time series interpolation). We experimentally illustrate that this framework can lead to a significant gain in terms of reconstruction performance, including w.r.t. the direct minimization of the variational formulation derived from the known generative model.
BibTeX
@misc{fablet2020joint,
  author = {Ronan Fablet and Lucas Drumetz and François Rousseau},
  title = {{Joint learning of variational representations and solvers for inverse problems with partially-observed data}},
  year = {2020},
  eprint = {2006.03653},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.03653},
}
Figure