Apprentissage d'a priori analyse

Gabriel Peyré, Jalal FadiliHAL 2011

TL;DR

This paper introduces a novel method for learning "analysis priors" for signal and image denoising. By optimizing dictionaries within an analysis framework, the authors demonstrate improved denoising performance compared to traditional methods, paving the way for more effective inverse problem solvers.

What Problem Does This Paper Address?

Denoising signals and images is a fundamental problem in signal processing, often tackled by solving variational optimization problems. These problems typically involve a data-fidelity term and a regularization term that enforces prior knowledge about the signal's structure. While "sparsity-promoting" priors have been highly successful, they are often formulated in a "synthesis" manner, where a signal is represented as a linear combination of dictionary atoms. This paper addresses the challenge of learning effective priors within the "analysis" framework, which is less explored but offers distinct advantages, particularly for capturing complex structures like discontinuities. The authors aim to generalize dictionary learning to this analysis setting, enabling the optimization of priors that are more directly suited for tasks like denoising and inverse problems.

Methods

The core contribution is a bilevel optimization problem formulation for learning analysis priors. This involves optimizing a dictionary D such that the solution x to a denoising problem (which uses D) minimizes a global objective function. To handle the non-differentiability of the standard L1 norm used for sparsity, the authors propose a smoothed version of the L1 norm, allowing for gradient-based optimization.

The paper then derives the sensitivity of the denoising solution x with respect to the dictionary D using the implicit function theorem. This is crucial for the bilevel optimization. They present algorithms for learning both:

  • Unstructured dictionaries: Where D is a general matrix.
  • Convolutional dictionaries: Where D is constrained to be translation-invariant, effectively learning a convolution kernel. This is achieved by leveraging properties of convolution operators and their adjoints, simplifying the gradient computation.

Datasets

The paper uses synthetic and natural 1D signals for evaluation.

  • Synthetic signals: Piecewise constant signals were used to demonstrate the ability to learn a Total Variation (TV) filter.
  • Natural signals: Lines extracted from the "Lena" image and the "Boat" image were used to learn convolutional priors.

Key Insights and Ideas

  • Analysis vs. Synthesis Priors: The paper highlights the distinction between synthesis (e.g., y ≈ Dx) and analysis (e.g., Dx is sparse) priors and proposes a unified framework for learning in the analysis setting.
  • Differentiable Priors for Learning: The need for differentiable priors to enable gradient-based dictionary learning is addressed by smoothing the L1 norm, a common technique in variational methods.
  • Bilevel Optimization for Priors: Framing dictionary learning as a bilevel optimization problem is a key insight, allowing the dictionary to be optimized for a specific task (denoising).
  • Convolutional Priors: Learning translation-invariant dictionaries (convolutional priors) offers a structured and efficient way to capture signal regularities, generalizing concepts like Total Variation.
  • Improved Denoising Performance: The learned priors, especially convolutional ones, demonstrate tangible improvements in denoising performance over standard TV regularization.

Key Results

  • Unstructured Dictionary Learning: The method successfully learned a Total Variation (TV) filter from piecewise constant signals, achieving denoising performance comparable to a hand-tuned TV filter. The energy of denoising E(D(t)) converged towards the energy of the TV dictionary E(Dtv,λ).
  • Convolutional Dictionary Learning: Learned convolutional priors improved denoising performance by approximately 0.8 dB on "Lena" lines and 0.3 dB on "Boat" lines compared to standard TV regularization. The convergence was faster with better initialization and structuring.

Limitations and Future Work

  • 1D Focus: The current work is limited to 1D signals.
  • Computational Cost: The bilevel optimization and gradient computations can be computationally intensive.
  • Local Minima: The optimization problem is non-convex, meaning the learned dictionary might converge to a local minimum.

The authors suggest several promising extensions:

  • Extension to Higher Dimensions: Applying the method to 2D images and potentially higher-dimensional data.
  • Learning Multiple Kernels: Extending the convolutional prior to learn multiple convolution kernels for more complex signal representations.
  • Other Inverse Problems: Applying the learned priors to other inverse problems like deconvolution and classification.

Summary updated

Abstract

National audience
BibTeX
@inproceedings{peyre2011apprentissage,
  author = {Gabriel Peyré and Jalal Fadili},
  title = {{Apprentissage d'a priori analyse}},
  booktitle = {HAL},
  year = {2011},
}
Figure