Noise insensitive volumetric fusion method for enhanced photoacoustic microscopy

Sihang Li, Hao Wu, Hongyu Zhang, Zhipeng Zhang, Q. Liu, Xianlin SongJournal of Biomedical Optics 2023 · DOI

TL;DR

This paper introduces a novel method for extending the depth-of-field (DoF) in optical-resolution photoacoustic microscopy (OR-PAM) by fusing multiple images captured at different focal planes. This technique significantly improves volumetric imaging capabilities without sacrificing lateral resolution, offering a cost-effective solution for enhanced biomedical imaging.

What Problem Does This Paper Address?

Photoacoustic imaging, which combines optical contrast with acoustic penetration, is a powerful tool for biomedical research. However, OR-PAM, a high-resolution variant, faces a significant challenge: a limited depth-of-field (DoF). This limitation arises from the tightly focused laser beam required for high resolution, meaning only a thin slice of the sample is in focus at any given time. Capturing a full 3D image thus requires extensive scanning, which is time-consuming and can be impractical for dynamic biological processes. Existing methods to extend DoF often involve complex and expensive setups. This paper aims to provide a more accessible and efficient solution by intelligently fusing data from multiple focal planes.

Methods

The core of the proposed method is a volumetric fusion technique that leverages a 3D modified Laplacian operator to accurately identify focused regions within multi-focus photoacoustic data. This operator quantifies the sharpness of the photoacoustic signal, highlighting areas with high-frequency variations indicative of being in focus.

The process involves several key steps:

  1. Initial Decision Map (IDM) Generation: The 3D modified Laplacian operator is applied to each voxel of the multi-focus images. For each block of voxels, the sum-modified Laplacian (SML) is calculated. By comparing the SML values between different focal planes, an initial decision map (IDM) is created, indicating which focal plane provides the sharpest image for each region.
  2. Consistency Verification and Smoothing: To address noise-induced errors in focus detection, a majority filter is used for consistency verification. This step refines the IDM by ensuring that the focus assignment for a block is consistent with its neighbors. Subsequently, a Gaussian filter is applied to smooth the map and reduce block artifacts, resulting in a final decision map (FDM).
  3. Volumetric Fusion: The FDM then guides a voxel-wise weighted-averaging process. This means that for each voxel in the final fused image, its intensity is a weighted combination of the corresponding voxels from the multi-focus images, with the weights determined by the FDM. This ensures that the sharpest information from each focal plane is preferentially included in the final image.

Datasets

The authors utilized simulated multi-focus 3D photoacoustic data generated using a virtual OR-PAM system. This simulation allowed for controlled experimentation with different noise levels and sample structures. Specifically, they simulated data for:

  • Vertically tilted fibers: Used to demonstrate the method's ability to extend DoF and preserve lateral resolution.
  • Vessels: Used to test the robustness and effectiveness of the method under varying noise conditions.

Four sets of multi-focus vessel data were generated at five different noise levels (ranging from no noise to SNR = 15 dB) to comprehensively evaluate the method's performance.

Key Insights and Ideas

  • Leveraging 3D Spatial Information: The use of a 3D modified Laplacian operator is crucial. Unlike 2D approaches, it considers the volumetric nature of the data, leading to more accurate focus detection in 3D space.
  • Noise Robustness is Key: The paper explicitly addresses noise sensitivity, a common issue in photoacoustic imaging. The combination of majority filtering and Gaussian smoothing effectively mitigates noise's impact on focus detection and fusion.
  • Cost-Effective DoF Extension: The method achieves a significant DoF extension (1.7x) without requiring additional hardware, making it a practical and accessible upgrade for existing OR-PAM systems.
  • Preservation of Lateral Resolution: A critical finding is that the proposed fusion method extends the DoF without sacrificing the lateral resolution achieved in the in-focus regions. This is vital for detailed structural imaging.

Key Results

  • The proposed method successfully extended the DoF of OR-PAM by a factor of 1.7 compared to single-focus imaging.
  • Lateral resolution was preserved within the focused regions of the fused images.
  • The method demonstrated robustness to noise, maintaining effective fusion even at SNR levels as low as 15 dB.
  • Quantitative evaluation using metrics like FWHM and SNR showed significant improvements in DoF and signal quality in the fused images compared to individual focal planes.
  • Comparison with two state-of-the-art 2D multi-focus image fusion methods (DTCWT and GFDF) showed superior performance of the proposed 3D volumetric fusion approach, particularly in preserving lateral resolution and handling noise.

Limitations and Future Work

  • Computational Cost: While cost-effective in terms of hardware, the fusion process, especially with many focal planes, can be computationally intensive. The paper mentions that for samples with intricate structures, estimating a focused boundary through simpler methods like FWHM quantification is difficult, suggesting the proposed method's advantage in such cases but also implying its complexity.
  • Iterative Fusion for Multiple Focuses: For datasets with more than two focal planes, the paper suggests an iterative pairwise fusion approach. While effective, this could further increase computational time. Future work could explore more efficient algorithms for fusing multiple focal planes simultaneously.
  • Application to Turbid Tissues: The authors note that simpler methods are not applicable to turbid biological tissues due to scattering and absorption variations. While their method is more robust, further investigation into its performance in highly scattering media might be beneficial.
  • Real-world Validation: The study primarily relies on simulated data. While the virtual OR-PAM is stated to be consistent with actual systems, validation with extensive experimental data from diverse biological samples would further strengthen the findings.

Summary updated

Abstract

The proposed method can efficiently extend the DoF of OR-PAM under different noise levels.
BibTeX
@article{li2023noise,
  author = {Sihang Li and Hao Wu and Hongyu Zhang and Zhipeng Zhang and Q. Liu and Xianlin Song},
  title = {{Noise insensitive volumetric fusion method for enhanced photoacoustic microscopy}},
  journal = {Journal of Biomedical Optics},
  year = {2023},
  doi = {10.1117/1.jbo.28.10.106501},
}
Figure