Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual Constraints

Shuhan Li, Dong Zhang, Xiaomeng Li, Chubin Ou, Lin An, Yanwu Xu, Kwang-Ting Cheng

TL;DR

TransPro is a novel 3D image-to-image translation framework that converts standard Optical Coherence Tomography (OCT) scans into high-quality Angiography (OCTA) scans without requiring expensive specialized hardware. By enforcing consistency between 3D volumes and 2D projection maps and utilizing vessel-aware semantic guidance, TransPro significantly outperforms existing methods in reconstructing retinal microvasculature.

What Problem Does This Paper Address?

Optical Coherence Tomography (OCT) is a standard, non-invasive tool for 3D retinal imaging. However, to see blood flow—crucial for diagnosing diseases like diabetic retinopathy—clinicians need OCT Angiography (OCTA). While powerful, OCTA hardware is expensive and sensitive to patient movement, requiring complex motion-tracking modules that limit its widespread adoption.

Deep learning offers a "software-only" alternative: translating readily available OCT scans into OCTA. Previous attempts at this translation have suffered from two main gaps:

  1. Dimensional Inconsistency: They often focus on either 2D slices (B-scans) or 2D projection maps, ignoring the 3D continuity of the retinal volume or the mathematical relationship between the 3D volume and its 2D projection.
  2. Vessel Blurring: Standard pixel-wise loss functions treat all pixels equally. Since vessels occupy a small fraction of the image compared to the background, models often produce blurry or disconnected vascular structures.

Methods

The authors introduce TransPro, a hybrid architecture centered around a 3D Conditional Generative Adversarial Network (GAN). The framework is built on two primary innovations:

  • Heuristic Contextual Guidance (HCG): This module ensures that the generated 3D volume is consistent with its 2D projection map. Since an OCTA projection map is physically generated by averaging pixel values along the Z-axis (depth), TransPro uses a pre-trained 2D generator to provide a "gold standard" projection that guides the 3D model, effectively acting as a regularizer against scanning artifacts and discontinuities.
  • Vessel Promoted Guidance (VPG): To prevent the model from ignoring fine vessels, the authors incorporate a pre-trained retinal vascular segmentation model. By comparing the segmentation masks of the generated images against the ground truth, the VPG module forces the network to prioritize the semantic accuracy of vessel morphology rather than just overall pixel intensity.
Fig. 2: The main ideas and contributions of this work. (a) illustrates the overall framework of the existing OCT to OCTA translation methods. (b) illustrates our proposed TransPro. Compared to existing methods, TransPro not only takes into considerations of critical vessel information via the Vessel-Promoted Guidance model but also introduces beneficial contexts via the Heuristic Contextual Guidance model.

The training process is multi-staged: first, the VPG and HCG modules are pre-trained and frozen, then they provide auxiliary losses to train the main 3D generator and its associated 2D and 3D discriminators.

Datasets

The model was evaluated on the OCTA-500 public dataset, which contains 500 pairs of 3D OCT and OCTA volumes. The researchers tested two specific subsets:

  • OCTA-3M: 200 volumes with a 3mm × 3mm field of view (304x304x640 pixels).
  • OCTA-6M: 300 volumes with a 6mm × 6mm field of view (400x400x640 pixels).

Key Insights and Ideas

  • The Projection Constraint: A key "aha moment" was realizing that the relationship between a 3D volume and a 2D projection map is a powerful, underutilized constraint. Enforcing this mathematical consistency helps the model maintain structural integrity across different views.
  • Semantic vs. Pixel-wise Loss: The paper demonstrates that for medical imaging, "looking right" (low MAE) isn't enough. By using a segmentation model as a loss function (VPG), the model learns the concept of a vessel, leading to much better clinical utility.
  • Hybrid Dimensionality: While the core task is 3D, leveraging 2D projections during training allows the model to benefit from the higher signal-to-noise ratio typically found in projection maps.

Key Results

TransPro achieved state-of-the-art results across both datasets. Compared to the baseline 3D pix2pix model on the OCTA-3M dataset, TransPro showed:

  • 11.4% improvement in Mean Absolute Error (MAE).
  • 40.8% reduction in Vessel Density Error (VDE), a metric specifically designed to measure clinical accuracy.
  • 14.0% boost in Vessel Density Correlation (VDC).

Qualitatively, as seen in Figure 5, TransPro produces much clearer vessel continuity and fewer artifacts than previous methods like AdjacentGAN.

Fig. 2: The main ideas and contributions of this work. (a) illustrates the overall framework of the existing OCT to OCTA translation methods. (b) illustrates our proposed TransPro. Compared to existing methods, TransPro not only takes into considerations of critical vessel information via the Vessel-Promoted Guidance model but also introduces beneficial contexts via the Heuristic Contextual Guidance model.

Crucially, in a blind test with an experienced ophthalmologist, the disease recognition results (identifying Diabetic Retinopathy and Choroidal Neovascularization) were identical between real OCTA scans and those translated by TransPro, achieving a 100% accuracy rate in the study sample.

Limitations and Future Work

While the results are impressive, the authors note that the model still struggles to reconstruct the very finest vascular details with perfect fidelity. Consequently, TransPro is currently viewed as a powerful screening and diagnostic aid rather than a total replacement for high-end OCTA hardware in all clinical scenarios.

Future work will focus on applying the TransPro framework to other 3D medical imaging tasks, exploring advanced data augmentation, and improving domain adaptation to make the model more robust across different scanner manufacturers.

Summary updated

Abstract

Optical Coherence Tomography Angiography (OCTA) is a crucial tool in the clinical screening of retinal diseases, allowing for accurate 3D imaging of blood vessels through non-invasive scanning. However, the hardware-based approach for acquiring OCTA images presents challenges due to the need for specialized sensors and expensive devices. In this paper, we introduce a novel method called TransPro, which can translate the readily available 3D Optical Coherence Tomography (OCT) images into 3D OCTA images without requiring any additional hardware modifications. Our TransPro method is primarily driven by two novel ideas that have been overlooked by prior work. The first idea is derived from a critical observation that the OCTA projection map is generated by averaging pixel values from its corresponding B-scans along the Z-axis. Hence, we introduce a hybrid architecture incorporating a 3D adversarial generative network and a novel Heuristic Contextual Guidance (HCG) module, which effectively maintains the consistency of the generated OCTA images between 3D volumes and projection maps. The second idea is to improve the vessel quality in the translated OCTA projection maps. As a result, we propose a novel Vessel Promoted Guidance (VPG) module to enhance the attention of network on retinal vessels. Experimental results on two datasets demonstrate that our TransPro outperforms state-of-the-art approaches, with relative improvements around 11.4% in MAE, 2.7% in PSNR, 2% in SSIM, 40% in VDE, and 9.1% in VDC compared to the baseline method. The code is available at: https://github.com/ustlsh/TransPro.
BibTeX
@article{li2024vessel,
  author = {Shuhan Li and Dong Zhang and Xiaomeng Li and Chubin Ou and Lin An and Yanwu Xu and Kwang-Ting Cheng},
  title = {{Vessel-Promoted OCT to OCTA Image Translation by Heuristic Contextual Constraints}},
  journal = {Medical Image Analysis},
  year = {2024},
  doi = {10.1016/j.media.2024.103311},
}
Figure