Controlling Neural Networks via Energy Dissipation

Michael Möller, Thomas Möllenhoff, Daniel CremersICCV 2019 · OpenReview

Abstract: The last decade has shown a tremendous success in solving various computer vision problems with the help of deep learning techniques. Lately, many works have demonstrated that learning-based approaches with suitable network architectures even exhibit superior performance for the solution of (ill-posed) image reconstruction problems such as deblurring, super-resolution, or medical image reconstruction. The drawback of purely learning-based methods, however, is that they cannot provide provable guarantees for the trained network to follow a given data formation process during inference. In this work we propose energy dissipating networks that iteratively compute a descent direction with respect to a given cost function or energy at the currently estimated reconstruction. Therefore, an adaptive step size rule such as a line-search, along with a suitable number of iterations can guarantee the reconstruction to follow a given data formation model encoded in the energy to arbitrary precision, and hence control the model's behavior even during test time. We prove that under standard assumptions, descent using the direction predicted by the network converges (linearly) to the global minimum of the energy. We illustrate the effectiveness of the proposed approach in experiments on single image super resolution and computed tomography (CT) reconstruction, and further illustrate extensions to convex feasibility problems.

Introduction: The "Black Box" Challenge in Inverse Problems

Deep learning has revolutionized image reconstruction, but it often operates as a "black box." In inverse problems—such as reconstructing a 3D CT scan from 2D X-rays or upscaling low-resolution images—purely learned networks are prone to overfitting. They may produce visually pleasing, sharp images that "hallucinate" details or, more dangerously, ignore critical anomalies (like small tumors) if those features were underrepresented in the training data.

The fundamental issue is the lack of provable guarantees. Standard networks lack a mechanism to ensure their outputs are consistent with the known physical data formation process.

Comparison of reconstruction methods on CT data. A purely learned network (third) misses a critical pathology (white blob) because it was not in the training set. The energy dissipating network (fourth) recovers it by respecting the data fidelity.

Energy Dissipating Networks (EDNs)

To bridge the gap between data-driven power and physical consistency, this paper introduces Energy Dissipating Networks (EDNs). Instead of a network that outputs a final image in a single pass, EDNs are designed to iteratively predict a descent direction for a specific energy function (e.g., the difference between the predicted image and actual measurements).

Key Technical Components

  • Projection Layer: The network's output passes through a specialized layer that mathematically guarantees it is a valid descent direction. This ensures every step reduces the reconstruction error (energy).
  • Adaptive Step Size: A standard backtracking line-search is employed to find the optimal step size, ensuring stability and monotonic convergence.
  • Lagged Training: The network is trained on the actual optimization paths it takes, iteratively updating the training distribution to include images generated by previous model versions.

Key Insights: Learned Flow Fields and Convergence

Unlike standard gradient descent, which follows a direct path to the nearest solution, EDNs learn a "flow field." This allows the network to navigate the energy landscape more intelligently, steering toward solutions that align with the characteristics of the training data while remaining anchored to the physical constraints.

Visualization of the learned gradient flow field. The network learns a path (blue) that converges to the solution subspace (red line) defined by the data formation model.

Furthermore, the authors provide a mathematical proof that, under standard assumptions, the descent direction predicted by the network converges linearly to the global minimum of the energy function.

Experimental Results

The researchers evaluated EDNs across three distinct domains:

  1. Single Image Super-Resolution: On the Set5 benchmark, EDNs achieved a PSNR of 31.16, significantly outperforming the baseline network's 29.94.
  2. CT Reconstruction: The method produced high-quality reconstructions in just 5-10 iterations, proving much faster than traditional regularized optimization techniques.
  3. Sudoku Solving: In a logical task, the EDN solved 52% of test puzzles, a massive improvement over the baseline network's 4% success rate, demonstrating the framework's versatility beyond image processing.

Conclusion

Energy Dissipating Networks offer a robust framework for integrating physical models into deep learning architectures. By forcing networks to respect the underlying physics of a problem, EDNs provide the safety and interpretability required for critical applications like medical imaging, without sacrificing the performance gains of modern neural networks.

Summary updated

Abstract

The last decade has shown a tremendous success in solving various computer vision problems with the help of deep learning techniques. Lately, many works have demonstrated that learning-based approaches with suitable network architectures even exhibit superior performance for the solution of (ill-posed) image reconstruction problems such as deblurring, super-resolution, or medical image reconstruction. The drawback of purely learning-based methods, however, is that they cannot provide provable guarantees for the trained network to follow a given data formation process during inference. In this work we propose energy dissipating networks that iteratively compute a descent direction with respect to a given cost function or energy at the currently estimated reconstruction. Therefore, an adaptive step size rule such as a line-search, along with a suitable number of iterations can guarantee the reconstruction to follow a given data formation model encoded in the energy to arbitrary precision, and hence control the model's behavior even during test time. We prove that under standard assumptions, descent using the direction predicted by the network converges (linearly) to the global minimum of the energy. We illustrate the effectiveness of the proposed approach in experiments on single image super resolution and computed tomography (CT) reconstruction, and further illustrate extensions to convex feasibility problems.
BibTeX
@inproceedings{moller2019controlling,
  author = {Michael Möller and Thomas Möllenhoff and Daniel Cremers},
  title = {{Controlling Neural Networks via Energy Dissipation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=omqbroCr9_5},
}
Figure