A Primal-dual algorithm for image reconstruction with ICNNs

Hok Shing Wong, Matthias J. Ehrhardt, Subhadip MukherjeearXiv 2024 · DOI · arXiv

Abstract

Image reconstruction is an ill-posed inverse problem typically addressed via variational frameworks. While Input-Convex Neural Networks (ICNNs) offer a powerful class of learned regularizers that guarantee global optimality, their nested, non-smooth structure makes them difficult to optimize using standard proximal methods. This paper presents a novel primal-dual algorithm that reformulates the ICNN regularizer into a set of convex constraints. By utilizing epigraphical projections for layer-wise activations, the proposed method avoids intractable proximal operators and achieves superior convergence speed and stability compared to subgradient-based approaches.

Introduction

Image reconstruction tasks, such as denoising or Computed Tomography (CT) scans, are fundamentally ill-posed. To recover high-quality images from degraded data, researchers employ a variational framework that minimizes a data fidelity term alongside a regularizer. Traditional hand-crafted regularizers like Total Variation (TV) often fail to capture the intricate textures of natural images. This has led to the adoption of learned regularizers, specifically Input-Convex Neural Networks (ICNNs).

ICNNs are designed such that the network output is a convex function of its input. This property is crucial as it ensures the overall optimization problem remains convex, allowing for global convergence. However, the nested architecture and non-smooth activation functions (e.g., ReLU) of ICNNs pose significant optimization challenges. Standard gradient methods are often slow or unstable, and the proximal operators required for more advanced algorithms are mathematically intractable for deep networks.

Methods

Constrained Convex Reformulation

The core contribution of this work is a Constrained Convex Reformulation of the ICNN-regularized objective. Rather than treating the ICNN as a single composite function $R(x)$, the authors "unroll" the network by introducing auxiliary variables for the activations of every layer.

Convexity property illustrated: the line segment between two points on the graph lies within the epigraph, a property leveraged by ICNNs.

Epigraphical Projections

The key innovation lies in transforming the network's equality constraints into inequality constraints based on the function's epigraph (the set of points lying on or above its graph). This transformation allows the use of the Primal-Dual Hybrid Gradient (PDHG) algorithm. While the proximal operator for the entire network is impossible to compute in closed form, the epigraphical projection for common activations like ReLU or Leaky ReLU is simple and can be computed exactly. This decoupling allows the optimization to handle the non-smoothness of each layer individually.

Experiments and Results

Datasets

The proposed method was evaluated on three imaging tasks:

  1. Denoising: 1,000 grayscale images from the FFHQ dataset.
  2. CT Reconstruction: 2,240 slices from the Mayo Clinic Low-dose CT Grand Challenge.
  3. Inpainting: Recovery of images from the FFHQ dataset with 30% random pixel removal.

Performance Metrics

The primal-dual approach demonstrated significant improvements in both speed and stability:

  • Convergence Speed: In denoising, the method reached visually satisfactory results in approximately 15 iterations, whereas subgradient methods remained noisy or required hundreds of iterations.
  • Efficiency: In inpainting tasks, the method achieved a 1.5x to 7.5x speedup over various subgradient descent configurations to reach target error thresholds.
  • Quality: When integrated into a bilevel training loop, the proposed solver yielded a Mean PSNR of 30.47, significantly higher than the 28.38 achieved by standard adversarial training.

Conclusion

This work demonstrates that reformulating ICNNs through epigraphical projections enables the use of efficient primal-dual algorithms for image reconstruction. By decoupling the layers, the algorithm achieves faster and more stable convergence than traditional methods. Future work will investigate coordinate-descent variants and alternative training strategies to better leverage the potential of deeper convex architectures.

Summary updated

Abstract

We address the optimization problem in a data-driven variational reconstruction framework, where the regularizer is parameterized by an input-convex neural network (ICNN). While gradient-based methods are commonly used to solve such problems, they struggle to effectively handle non-smooth problems which often leads to slow convergence. Moreover, the nested structure of the neural network complicates the application of standard non-smooth optimization techniques, such as proximal algorithms. To overcome these challenges, we reformulate the problem and eliminate the network's nested structure. By relating this reformulation to epigraphical projections of the activation functions, we transform the problem into a convex optimization problem that can be efficiently solved using a primal-dual algorithm. We also prove that this reformulation is equivalent to the original variational problem. Through experiments on several imaging tasks, we show that the proposed approach not only outperforms subgradient methods and even accelerated methods in the smooth setting, but also facilitates the training of the regularizer itself.
BibTeX
@misc{wong2024primal,
  author = {Hok Shing Wong and Matthias J. Ehrhardt and Subhadip Mukherjee},
  title = {{A Primal-dual algorithm for image reconstruction with ICNNs}},
  year = {2024},
  eprint = {2410.12441},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.12441},
}
Figure