Low Photon Budget Phase Retrieval with Perceptual Loss Trained Deep Neural Networks

Mo Deng, Alexandre Goy, Shuai Li, Kwabena Arthur, George BarbastathisarXiv 2019 · DOI · arXiv

Abstract

Deep neural networks (DNNs) are efficient solvers for ill-posed problems in computational imaging. Training these networks requires selecting an appropriate loss function. While the negative Pearson correlation coefficient (NPCC) has previously shown success in photon-starved phase retrieval, we demonstrate that a perceptual loss function significantly improves reconstruction quality. By prioritizing feature recognizability over pixel-wise accuracy, our method recovers sharp details in conditions fundamentally limited by strong Poisson noise.

Introduction

Phase retrieval is a classic challenge where only the intensity of light is measured, and the phase information is lost. In "low photon budget" scenarios, such as specialized microscopy or X-ray crystallography, the signal is often overwhelmed by Poisson (shot) noise. Traditional optimization methods struggle with these conditions, and standard DNN training metrics like Mean Squared Error (MSE) often produce blurry, over-smoothed images. This research addresses the problem by training a Phase Extraction Neural Network (PhENN) using a loss function that mimics human visual perception.

Methodology

The reconstruction process involves two stages. First, a physics-inspired preprocessing step uses a single iteration of the Gerchberg-Saxton algorithm to create an initial "approximant." This approximant is then refined by PhENN.

The core innovation is the use of Perceptual Loss. Instead of minimizing pixel differences, the network's output and the ground truth are passed through a pre-trained VGG16 classification network. The loss is calculated based on the difference between feature maps at the relu 2-2 layer. This approach trains the network to produce images that are recognizable to high-level classifiers, preserving semantic details like edges and textures.

Training architecture for PhENN using perceptual loss through a feature-extracting network.

Datasets and Calibration

The model was trained on 9,500 images from the ImageNet database, with 450 images used for validation and 50 for testing. Because perceptual loss is non-linear, the authors applied a final third-degree polynomial calibration to the reconstructed phase values. This step ensures that the final output is not only visually sharp but also quantitatively accurate.

Key Results

At extremely low light levels (1 and 10 photons per pixel), the perceptual loss model produced significantly sharper images than the previous state-of-the-art NPCC model. Details that were previously blurred out—such as vertical posts or fine fabric patterns—became clearly identifiable. The results demonstrate that recognizability is a powerful prior for solving ill-posed imaging problems.

Visual comparison of reconstructions at 10 and 1 photon levels, showing the superiority of perceptual loss over NPCC.

Discussion and Limitations

Despite its success, the perceptual loss model introduces systematic grid-like artifacts at very low photon levels. These artifacts occur at a characteristic frequency ($\nu_c = 0.25$ pixel⁻¹), corresponding to a 4-pixel period. The researchers traced this to the internal structure of the VGG16 network, which exhibits a "non-smooth" loss landscape at this specific frequency. Future work will involve investigating the weights of pre-trained networks to mitigate these artifacts while retaining the superior detail provided by perceptual loss.

Summary updated

Abstract

Deep neural networks (DNNs) are efficient solvers for ill-posed problems and have been shown to outperform classical optimization techniques in several computational imaging problems. DNNs are trained by solving an optimization problem implies the choice of an appropriate loss function, i.e., the function to optimize. In a recent paper [A. Goy \textit{et al.}, Phys. Rev. Lett. 121(24), 243902 (2018)], we showed that DNNs trained with the negative Pearson correlation coefficient as the loss function are particularly fit for photon-starved phase retrieval problems, which are fundamentally affected by strong Poison noise. In this paper we demonstrate that the use of a perceptual loss function significantly improves the reconstructions.
BibTeX
@misc{deng2019low,
  author = {Mo Deng and Alexandre Goy and Shuai Li and Kwabena Arthur and George Barbastathis},
  title = {{Low Photon Budget Phase Retrieval with Perceptual Loss Trained Deep Neural Networks}},
  year = {2019},
  eprint = {1906.05687},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.05687},
}
Figure