Deep feature loss to denoise OCT images using deep neural networks

Maryam Mehdizadeh, Cara MacNish, Di Xiao, David Alonso‐Caneiro, Jason Kugelman, Mohammed BennamounJournal of Biomedical Optics 2021 · DOI

TL;DR

This paper introduces a novel "deep feature loss" approach to denoise Optical Coherence Tomography (OCT) images, aiming to preserve fine details crucial for clinical interpretation. By leveraging features from a pre-trained VGG network as a loss function, they achieve perceptually sharper images compared to traditional denoising methods, though with a slight trade-off in smoothness.

What Problem Does This Paper Address?

Optical Coherence Tomography (OCT) is a vital imaging technique for visualizing the internal structures of the eye, particularly the retina. However, OCT images are inherently plagued by speckle noise, which significantly degrades image quality, obscures subtle but critical anatomical details (like retinal layer boundaries), and hinders accurate clinical diagnosis and automated analysis. Existing denoising methods often struggle to strike a balance between noise reduction and preserving these fine features, sometimes leading to overly smoothed or artifact-ridden images. This paper addresses the challenge of developing a denoising method that effectively reduces speckle noise while enhancing perceptual sharpness and preserving essential anatomical details in OCT images.

Methods

The core innovation of this paper lies in its use of a "deep feature loss" function for training a Convolutional Neural Network (CNN) for OCT denoising. Specifically, they employ a well-known denoising network architecture called DnCNN and augment it with a pre-trained VGG-16 network. Instead of relying solely on traditional pixelwise loss functions (like L1 or L2, which measure the direct difference between pixels), they use the VGG-16 network to extract features from both the noisy and denoised images. The difference between these extracted deep features then forms the perceptual loss. This approach is inspired by the idea that features learned by deep networks for tasks like image classification can better capture perceptual similarities than simple pixel differences.

The authors also explore the contribution of individual layers within the VGG-16 network to this deep feature loss, hypothesizing that not all levels of feature abstraction might be equally beneficial for OCT denoising. The VGG-16 network's weights are kept fixed during training, acting solely as a feature extractor for the loss calculation.

Datasets

The study utilized a dataset of foveal-centered OCT retinal scans from 226 children with normal vision. For training, 51 OCT image pairs (noisy and averaged) were randomly selected. A separate set of 20 OCT image pairs was used for testing and validation. The images were divided into overlapping patches for processing. No new benchmark datasets were introduced; the study used an existing dataset.

Key Insights and Ideas

  • Deep features capture perceptual similarity better than pixels: The fundamental idea is that the "essence" of an image, in terms of what humans perceive as similar, is better represented by high-level features learned by deep networks than by raw pixel values.
  • Not all VGG layers are created equal for denoising: The paper investigates the impact of using features from different layers of the VGG-16 network. This suggests that specific levels of abstraction within the VGG network might be more effective for OCT denoising than others, or the network as a whole.
  • Trade-off between smoothness and sharpness: The study highlights that while traditional pixelwise losses (L1/L2) can achieve high PSNR (indicating smoothness), they often lead to blurry images. Deep feature losses, while potentially yielding lower PSNR, result in perceptually sharper images with better preservation of fine details.
  • PSNR is insufficient for evaluating denoising quality: The authors emphasize that relying solely on PSNR can be misleading, as it doesn't fully capture the perceptual quality and sharpness of the denoised image, which are critical for clinical applications.

Key Results

  • Improved Perceptual Sharpness: The deep feature loss approach, particularly using specific layers of the VGG-16 network (e.g., DnCNN-Conv5-3), consistently outperformed traditional L1 and L2 losses in perceptual sharpness metrics like PSI, JNB, and S3. For instance, PSI values for VGG-based networks were around 0.30-0.33, compared to 0.21 for L1/L2.
  • Better Preservation of Fine Features: Qualitative analysis and visual inspection showed that VGG-based networks preserved retinal layer boundaries and faint features (like the ILM layer) more effectively, avoiding the blurring and oversmoothing seen with L1/L2 losses.
  • Lower PSNR: As expected, the deep feature loss methods generally achieved lower PSNR values compared to L1 and L2 losses, indicating a trade-off between smoothness and sharpness.
  • Individual VGG layers show promise: Some individual VGG layers (e.g., Conv4-3, Conv5-3) showed particularly good results in terms of sharpness and feature preservation, suggesting that a carefully selected subset of VGG layers could be more effective than the entire network.

Limitations and Future Work

  • Dataset Specificity: The study was conducted on OCT images from a single instrument. Future work should involve training and testing on datasets from different OCT devices to ensure the generalizability of the proposed method.
  • Layer-wise vs. Filter-wise Analysis: The current work focuses on individual layers of the VGG network. Further research could explore the contribution of individual filters within these layers for more granular optimization.
  • Expert Validation: The study relied on quantitative metrics and visual inspection. Incorporating expert clinician evaluations would provide a more robust assessment of the clinical utility of the denoising methods.
  • Optimizing Layer Weights: The authors suggest exploring weighting schemes for combining features from multiple VGG layers to further optimize performance.
  • New Metrics: The paper advocates for the use of a broader range of perceptual metrics beyond PSNR for evaluating denoising performance.

Summary updated

Abstract

We demonstrate the potential of deep feature loss in denoising OCT images. Our preliminary findings suggest research directions for further investigation.
BibTeX
@article{mehdizadeh2021deep,
  author = {Maryam Mehdizadeh and Cara MacNish and Di Xiao and David Alonso‐Caneiro and Jason Kugelman and Mohammed Bennamoun},
  title = {{Deep feature loss to denoise OCT images using deep neural networks}},
  journal = {Journal of Biomedical Optics},
  year = {2021},
  doi = {10.1117/1.jbo.26.4.046003},
}
Figure