Learning to Sense for Coded Diffraction Imaging

Rakib Hyder, Zikui Cai, M. Salman Asif

TL;DR

This paper introduces a novel approach to improve image reconstruction in coded diffraction imaging by learning optimal illumination patterns. By framing phase retrieval as an unrolled neural network and optimizing the illumination patterns through backpropagation, the method achieves superior image quality with a fixed computational cost, even with limited training data.

What Problem Does This Paper Address?

Coded diffraction imaging (CDI) is a technique used in various fields, from X-ray crystallography to microscopy, to reconstruct an object's image from its intensity-only measurements. A core challenge in CDI is the "phase retrieval problem": recovering a complex-valued signal from its magnitude-only Fourier transform. Traditional methods often rely on iterative algorithms that can be computationally expensive and sensitive to the choice of illumination patterns. This paper addresses the problem of designing effective illumination patterns that can significantly enhance the performance of these iterative recovery algorithms, particularly when computational resources are limited.

Methods

The authors propose a data-driven approach to learn illumination patterns for CDI. Their method leverages an unrolled neural network formulation of a classical alternating minimization (AltMin) algorithm for phase retrieval. Instead of learning the entire reconstruction network, they fix the iterative recovery algorithm (AltMin) and its number of iterations (K), treating each iteration as a layer in a fixed-depth network. The core innovation lies in learning the illumination patterns themselves.

During training, the algorithm takes a set of training images and uses the unrolled AltMin network to reconstruct them with an initial set of illumination patterns. The reconstruction error (difference between the reconstructed and ground truth images) is then backpropagated to update the illumination patterns. This process optimizes the patterns to yield the best possible reconstruction from the fixed AltMin algorithm. At inference time, the learned illumination patterns are fixed, and the unrolled network performs a single forward pass for reconstruction, ensuring a fixed computational cost. The illumination patterns are constrained to be physically realizable (values between 0 and 1) using a sigmoid function.

Datasets

The paper utilizes several standard image datasets for training and evaluation:

  • MNIST digits
  • Fashion MNIST
  • CIFAR10
  • SVHN
  • CelebA

For training, they typically use a small subset of these datasets (e.g., 32 or 128 images), demonstrating the method's effectiveness with limited data. For testing, a larger set of images (e.g., 1000 images) from the same datasets is used.

Key Insights and Ideas

  • Learning the "Sensor" is Key: The paper highlights that optimizing the sensing mechanism (illumination patterns) can be more impactful than solely focusing on improving the reconstruction algorithm, especially when the algorithm's complexity is fixed.
  • Data Efficiency Matters: The ability to learn effective illumination patterns from a small number of training images is a significant advantage, making the approach practical for real-world scenarios where large labeled datasets might be scarce.
  • Generalization is Robust: Learned patterns show good generalization across different datasets and even perform well with algorithms they were not explicitly trained for, indicating a robust design.
  • Fixed Computational Cost: By unrolling a fixed-iteration iterative algorithm, the inference time is predictable and constant, offering a significant advantage over methods that might have variable or higher computational demands.
  • Zero Initialization Suffices: Surprisingly, the method performs well with simple zero initialization for the signal estimate, avoiding the need for more complex spectral initialization techniques.

Key Results

  • Superior Reconstruction Quality: The learned illumination patterns consistently outperform random patterns across various datasets and numbers of patterns (T=2, 3, 4, 8), achieving significantly higher Peak Signal-to-Noise Ratios (PSNR). For example, with T=4 patterns, PSNRs for learned patterns often exceed 100 dB, whereas random patterns struggle to reach 30 dB on some datasets.
  • Efficiency and Speed: The proposed method achieves high-quality reconstructions at a fixed computational cost, often outperforming other methods in terms of both accuracy and runtime. For instance, the learned patterns enable high PSNR with fewer iterations compared to random patterns.
  • Robustness to Noise: The learned patterns demonstrate reasonable reconstruction quality even when subjected to Gaussian and Poisson noise, indicating stability. Training with noise further improves performance under noisy conditions.
  • Generalization to Shifted/Rotated Images: The learned patterns successfully recover images that are shifted or rotated relative to the training data, showcasing their robustness to variations not explicitly seen during training.

Limitations and Future Work

  • Fixed Iteration Count: While a strength for computational cost, the fixed number of iterations (K) might limit ultimate reconstruction quality if a higher number of iterations is truly needed for a specific problem. The paper notes a trade-off between K and accuracy/runtime.
  • Dataset Dependence: While patterns generalize well, they are still learned from specific datasets. Future work could explore more advanced techniques for even broader generalization or domain adaptation.
  • Algorithm Specificity: The learned patterns are optimized for the AltMin algorithm. While they show some benefit for other algorithms, further research could investigate learning patterns tailored to different recovery algorithms or even learning both patterns and algorithm parameters.

Summary updated

BibTeX
@inproceedings{hyder2022learning,
  author = {Rakib Hyder and Zikui Cai and M. Salman Asif},
  title = {{Learning to Sense for Coded Diffraction Imaging}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=3rRNz-viRfa},
}
Figure