High-throughput molecular imaging via deep learning enabled Raman spectroscopy

Conor C. Horgan, Magnus Jensen, Anika Nagelkerke, Jean-Phillipe St-Pierre, Tom Vercauteren, Molly M. Stevens, Mads S. BergholtarXiv 2020 · DOI · OpenReview

TL;DR

Researchers have developed "DeepeR," a deep learning framework that dramatically accelerates Raman spectroscopy for molecular imaging. By combining denoising and super-resolution techniques, DeepeR achieves up to a 160x speedup, enabling high-resolution, high-signal-to-noise imaging of cells in under a minute, and can even be adapted for tissue imaging.

What Problem Does This Paper Address?

Raman spectroscopy is a powerful, non-destructive technique for molecular imaging, offering label-free contrast. However, its practical application is severely hampered by slow data acquisition. Capturing a high-quality Raman spectrum from each pixel in an image can take seconds, meaning imaging cells or tissues can take hours. This slowness prevents its use in high-throughput applications, such as screening or rapid diagnostics. While advanced Raman techniques exist, they often require complex and expensive hardware. This paper aims to overcome the speed limitations of conventional Raman spectroscopy using deep learning.

Methods

The core of this work is the "DeepeR" framework, which comprises two main deep learning components:

  • Raman Spectral Denoising: A 1D Residual UNet (ResUNet) is trained to reconstruct high-signal-to-noise (SNR) Raman spectra from low-SNR inputs. This effectively "cleans up" noisy spectra, preserving crucial biochemical information.
  • Hyperspectral Image Super-Resolution: A Residual Channel Attention Network (RCAN) is adapted to perform super-resolution on hyperspectral Raman images. This network learns to reconstruct high-spatial-resolution images from lower-resolution inputs.

These two components can be used sequentially. First, denoising improves the SNR of low-acquisition-time spectra, and then super-resolution increases the spatial resolution. The framework also leverages transfer learning to adapt models trained on large datasets to new, smaller datasets, such as those from tissue imaging.

Datasets

  • Cellular Imaging Dataset: Used for training the denoising network. It consists of 172,312 pairs of low-SNR (0.1s integration) and high-SNR (1s integration) hyperspectral Raman spectra from MDA-MB-231 breast cancer cells.
  • Hyperspectral Raman Images Dataset: Used for training the super-resolution network. This dataset comprises 169 hyperspectral images, representing approximately 1.4 million Raman spectra (equivalent to 389 hours of acquisition), also from MDA-MB-231 breast cancer cells.
  • Tissue Imaging Dataset: A smaller, unrelated dataset of hyperspectral Raman images of tissue-engineered cartilage constructs was used to demonstrate the generalizability of the framework through transfer learning.

Key Insights and Ideas

  • Deep Learning for Signal Reconstruction: The paper demonstrates that deep learning can effectively learn the underlying structure of Raman spectra, enabling superior denoising compared to traditional methods like Savitzky-Golay filtering.
  • Synergy of Denoising and Super-Resolution: Combining spectral denoising with spatial super-resolution is key to achieving dramatic speedups while maintaining image quality and preserving molecular information.
  • Transfer Learning for Domain Adaptation: The ability to transfer learned models from one type of biological sample (cells) to another (tissue) without extensive retraining is a significant advantage, especially when dealing with limited data.
  • Hardware-Agnostic Speedup: DeepeR offers a software-based solution to accelerate Raman imaging, potentially making high-throughput applications accessible without requiring expensive hardware upgrades.

Key Results

  • Denoising Performance: The 1D ResUNet achieved a 9x improvement in mean squared error (MSE) over the best-performing Savitzky-Golay filter for Raman spectral denoising.
  • Super-Resolution Performance: The hyperspectral super-resolution network achieved robust 2-4x image reconstruction, leading to a 4-16x reduction in imaging time.
  • Combined Speedup: By combining denoising and super-resolution, the framework achieved overall Raman imaging speed-ups of up to 160x, enabling cellular imaging in under one minute.
  • Information Preservation: The methods successfully preserved crucial biochemical information, as validated by Vertex Component Analysis (VCA) and pixel classification accuracy, outperforming traditional methods.
  • Transfer Learning Success: Transfer learning enabled effective super-resolution on a tissue dataset, demonstrating the framework's adaptability to new domains.

Limitations and Future Work

  • Fine Detail Reconstruction: The authors acknowledge that the framework may not accurately reconstruct very fine (single-pixel) details, making it potentially unsuitable for very high-resolution imaging of small, complex specimens. This is analogous to limitations in single-image super-resolution for RGB images.
  • Dataset Size for Transfer Learning: While transfer learning is effective, applications with substantially different spatial and spectral features will still require a sufficiently large dataset for optimal performance.
  • Future Work:
    • Expanding the depth and breadth of the hyperspectral Raman dataset to improve performance and generalizability.
  • Developing a joint denoising and super-resolution neural network for potentially improved performance.
  • Exploring Generative Adversarial Networks (GANs) for hyperspectral image super-resolution, which could offer further improvements but require significant computational resources.
  • Adapting the framework for other spectral imaging techniques (e.g., FT-IR, mass spectrometry) and for tasks like autofluorescence removal and segmentation.
  • Large-scale prospective validation to ensure diagnostic or scientific decisions match those made with high-resolution data.

Summary updated

Abstract

Raman spectroscopy enables non-destructive, label-free imaging with unprecedented molecular contrast but is limited by slow data acquisition, largely preventing high-throughput imaging applications. Here, we present a comprehensive framework for higher-throughput molecular imaging via deep learning enabled Raman spectroscopy, termed DeepeR, trained on a large dataset of hyperspectral Raman images, with over 1.5 million spectra (400 hours of acquisition) in total. We firstly perform denoising and reconstruction of low signal-to-noise ratio Raman molecular signatures via deep learning, with a 9x improvement in mean squared error over state-of-the-art Raman filtering methods. Next, we develop a neural network for robust 2-4x super-resolution of hyperspectral Raman images that preserves molecular cellular information. Combining these approaches, we achieve Raman imaging speed-ups of up to 160x, enabling high resolution, high signal-to-noise ratio cellular imaging in under one minute. Finally, transfer learning is applied to extend DeepeR from cell to tissue-scale imaging. DeepeR provides a foundation that will enable a host of higher-throughput Raman spectroscopy and molecular imaging applications across biomedicine.
BibTeX
@misc{horgan2020high,
  author = {Conor C. Horgan and Magnus Jensen and Anika Nagelkerke and Jean-Phillipe St-Pierre and Tom Vercauteren and Molly M. Stevens and Mads S. Bergholt},
  title = {{High-throughput molecular imaging via deep learning enabled Raman spectroscopy}},
  year = {2020},
  eprint = {2009.13318},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.13318},
}
Figure