Optimal transport-based dictionary learning and its application to Euclid-like Point Spread Function representation

Morgan A. Schmitz, Matthieu Heitz, Fred-Maurice Ngolè-Mboula, Nicolas Bonneel, David Cœurjolly, Marco Cuturi, Gabriel Peyré, Jean‐Luc Starck2017 · DOI

TL;DR

This paper introduces a novel dictionary learning method that leverages Optimal Transport theory to learn non-linear representations of data. By reconstructing data as Wasserstein barycenters of learned "atoms," the method captures complex geometric warping, outperforming traditional linear methods like PCA and NMF, particularly in representing the chromatic variations of telescope Point Spread Functions (PSFs).

What Problem Does This Paper Address?

Feature learning, or representation learning, is crucial for extracting meaningful information from raw data, especially for high-dimensional datasets like images. Traditional dictionary learning methods often rely on linear combinations of dictionary "atoms" to reconstruct data. While effective, this linear assumption can fail to capture complex, non-linear relationships present in many real-world phenomena. This is particularly true in astronomical imaging, where the Point Spread Function (PSF) – the blurring kernel of a telescope – exhibits complex chromatic (wavelength-dependent) and spatial variations. Accurately modeling these PSF variations is critical for precise measurements in surveys like ESA's Euclid mission, as they are a major source of systematic error in galaxy shape measurements. Existing methods struggle to capture these non-linear chromatic variations effectively.

Methods

The core innovation of this paper is the replacement of the standard linear reconstruction in dictionary learning (X ≈ DA) with a Wasserstein barycenter formulation. Instead of a simple matrix multiplication, the data is reconstructed as a weighted average of dictionary atoms in the "Wasserstein space." This space is defined by the Wasserstein distance, which naturally accounts for geometric transformations between data distributions (measures).

The authors adapt recent advancements in computing approximate Wasserstein distances using an entropy penalty term and the Sinkhorn algorithm. This allows for the computation of Wasserstein barycenters, which are then used as the reconstruction mechanism. The learning process involves minimizing an energy function that penalizes the difference between the original data and its Wasserstein barycenter reconstruction. This minimization is achieved through gradient descent, utilizing automatic differentiation to compute gradients for both the dictionary atoms and their corresponding weights.

Datasets

The paper uses two primary datasets for evaluation:

  • Toy Example: A set of discretized Gaussian distributions, slightly translated on a grid. This dataset is used to demonstrate the method's ability to capture non-linear transformations compared to PCA and NMF.
  • Euclid-like PSFs: Simulated Point Spread Functions (PSFs) for the ESA's Euclid mission, exhibiting chromatic variations across a range of wavelengths (550nm to 900nm). This dataset is crucial for showcasing the method's effectiveness in modeling complex, non-linear, wavelength-dependent phenomena.

Key Insights and Ideas

  • Non-linear Representation Power: The Wasserstein distance inherently captures geometric warping, allowing the learned representation to model non-linear relationships between dictionary atoms and data, which linear methods miss.
  • Wasserstein Barycenters as a Reconstruction Tool: Using Wasserstein barycenters for reconstruction provides a principled way to combine dictionary atoms that respects the underlying geometric structure of the data.
  • Entropy Regularization for Practicality: The use of entropy regularization in Wasserstein distance computation makes the method computationally feasible for practical applications, enabling its use in feature learning.
  • Capturing Chromatic PSF Variations: The method successfully models the complex chromatic variations of PSFs, a significant challenge for astronomical surveys. The learned atoms closely resemble the extreme wavelength PSFs, and the reconstructions are more accurate than those from linear methods.
  • Blurring Effect of Entropy: The entropy term, while enabling computation, introduces a slight blurring effect in the barycenters. This is acknowledged and can be controlled by the entropy parameter, though very small values can lead to numerical instability.

Key Results

  • Toy Example: The proposed method (WDL) successfully reconstructs translated Gaussian distributions using only two atoms, outperforming PCA and NMF, which produce less accurate reconstructions and non-physical (negative) values.
  • Euclid-like PSFs: The method learns atoms that closely resemble the PSFs at extreme wavelengths. Reconstructions of the PSFs are significantly more accurate than those achieved by NMF, particularly for intermediate wavelengths where chromatic warping is more pronounced. The reconstruction error for WDL was 1.71 × 10⁻³, compared to 2.62 × 10⁻³ for NMF.
  • Qualitative Findings: The learned atoms and reconstructions from the WDL method are visually more coherent and physically plausible than those from PCA and NMF, especially for the PSF data.

Limitations and Future Work

  • Computational Cost: While improved by entropy regularization, Wasserstein barycenter computation can still be computationally intensive, especially for very large datasets or high-dimensional data.
  • Local Minima: Like NMF, the dictionary learning process is non-convex, meaning the optimization can converge to local minima. The authors mitigated this by relaunching the optimization multiple times.
  • Generalization to More Atoms: The paper focuses on learning two atoms for simplicity. While the authors state generalization to more atoms is immediate, further investigation into the scalability and performance with a larger number of atoms would be beneficial.
  • Future Work: The authors suggest that this approach could be extended to more complex scenarios, such as modeling polychromatic PSFs by casting the decomposition into monochromatic components as an Optimal Transport problem. The method's ability to capture spatial variations in PSFs, in addition to chromatic ones, is also a promising avenue.

Summary updated

Abstract

International audience
BibTeX
@misc{schmitz2017optimal,
  author = {Morgan A. Schmitz and Matthieu Heitz and Fred-Maurice Ngolè-Mboula and Nicolas Bonneel and David Cœurjolly and Marco Cuturi and Gabriel Peyré and Jean‐Luc Starck},
  title = {{Optimal transport-based dictionary learning and its application to Euclid-like Point Spread Function representation}},
  year = {2017},
  doi = {10.1117/12.2270641},
}
Figure