Perfusion Quantification from Endoscopic Videos: Learning to Read Tumor Signatures

Sergiy Zhuk, Jonathan P. Epperlein, Rahul Nair, Seshu Tirupathi, Pol Mac Aonghusa, Ronan A. Cahill, Donal O'SheaarXiv 2020 · arXiv

Abstract

This paper presents a hybrid approach combining bio-physical modeling with machine learning to identify cancerous tissue in real-time during surgery. By analyzing the "wash-in" and "wash-out" kinetics of Indocyanine Green (ICG) fluorescent dye in endoscopic videos, the system distinguishes between healthy, benign, and malignant tissues. Achieving 95% patient-level accuracy, this data-efficient method provides a way to interpret unique vascular signatures of tumors intra-operatively.

Introduction

During cancer surgery, distinguishing between malignant tumors, benign growths, and healthy tissue is critical but challenging. Surgeons currently use Indocyanine Green (ICG), a fluorescent dye that highlights blood flow (perfusion). However, the human eye is poorly suited to tracking subtle, complex changes in fluorescence intensity across both space and time over several minutes. While deep learning could theoretically solve this, surgical datasets are often too small to train massive neural networks effectively. There is a need for a data-efficient way to interpret these perfusion patterns—essentially a way to help surgeons "read" the unique vascular signature of a tumor intra-operatively.

Methodology

The authors propose a method that bridges the gap between raw video data and clinical decision-making through a three-step pipeline:

  1. Fluorescence Intensity Estimation: Near-Infrared (NIR) intensity is extracted from specific Regions of Interest (ROIs) in multispectral endoscopic videos, serving as a proxy for ICG concentration.
  2. Bio-physical Modeling: Instead of raw pixels, the system fits time-series data to a second-order linear system model. This captures the physics of "wash-in" and "wash-out" dynamics. Features extracted include 3EXP (coefficients from differential equations) and TTP (Time-To-Peak metrics like slope and peak time).
  3. Classification: These engineered features are fed into a Gradient Boosted Tree (XGBoost) classifier. By using physics-informed features, the model requires significantly less training data than standard deep learning approaches.
Overview of the proposed three-step pipeline: intensity estimation, biophysical modeling, and region classification.

Results and Key Insights

The system was evaluated using a Leave-One-Out (LOO) cross-validation scheme on a dataset of 20 colorectal cancer videos (435 samples). The model exploits the fact that cancer-induced blood vessels (angiogenesis) are "leaky" and disorganized, leading to distinct "pooling" and drainage patterns that differ from healthy tissue.

Intensity curves for different regions (A) and the resulting classification overlay (C) showing high confidence in cancer detection.

Key Performance Metrics:

  • Patient-Level Accuracy: 95% (19 out of 20 cases correctly classified).
  • Sensitivity: 100% (The model correctly identified every cancerous case).
  • Specificity: 91.67%.
  • ROI-Based Accuracy: 86.38%.

A critical insight was the synergy between bio-physical "3EXP" features and traditional "TTP" features; combining them significantly boosted performance. Additionally, the authors handled patient variability by normalizing rate features against a patient’s own healthy tissue, using the patient as their own control group.

Conclusion and Future Work

This research demonstrates that physics-informed machine learning can achieve high diagnostic accuracy even with limited surgical datasets. While the results are promising, the current pipeline assumes pre-stabilized video, which is a challenge in real-world handheld surgery. Future work will focus on validating the model on larger cohorts, integrating it into surgical hardware for real-time augmented reality overlays, and determining if these vascular signatures are consistent across other types of solid tumors.

Summary updated

BibTeX
@misc{zhuk2020perfusion,
  author = {Sergiy Zhuk and Jonathan P. Epperlein and Rahul Nair and Seshu Tirupathi and Pol Mac Aonghusa and Ronan A. Cahill and Donal O'Shea},
  title = {{Perfusion Quantification from Endoscopic Videos: Learning to Read Tumor Signatures}},
  year = {2020},
  eprint = {2006.14321},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.14321},
}
Figure