Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach

Francesca Manni, Fons van der Sommen, Himar Fabelo, Svitlana Zinger, Caifeng Shan, Erik Edström, Adrian Elmi Terander, Samuel Ortega, Gustavo Marrero Callicó, Peter H. N. de With

TL;DR

This paper introduces a novel 3D-2D hybrid Convolutional Neural Network (CNN) approach for improved identification of glioblastoma tumors during surgery using hyperspectral imaging (HSI). By jointly analyzing spectral and spatial information from HSI data, their method achieves an impressive 80% accuracy in classifying tumor, healthy tissue, and blood vessels, outperforming existing state-of-the-art techniques.

What Problem Does This Paper Address?

Surgical resection is the primary treatment for glioblastoma (GBM), a highly aggressive brain tumor. While complete tumor removal (gross total resection) significantly improves patient prognosis, accurately identifying tumor boundaries during surgery is challenging. Tumors often infiltrate surrounding healthy brain tissue, making visual identification difficult. Inaccurate identification can lead to either leaving residual tumor (increasing recurrence risk) or resecting too much healthy tissue (causing neurological deficits). Current methods like intraoperative MRI have limitations in resolution and workflow. Histological analysis of biopsies is too slow for real-time surgical guidance. Hyperspectral imaging (HSI) offers a promising non-invasive solution by capturing detailed spectral information from tissue, but its high dimensionality and the need to integrate spectral and spatial data for accurate classification have been significant hurdles.

Methods

The core of this work is a 3D-2D hybrid CNN architecture designed to leverage both the spectral and spatial information present in hyperspectral images.

  • HSI Data Acquisition: The study utilized an existing intraoperative HSI system that captures data in the visible and near-infrared (VNIR) spectral range (400-1000 nm), resulting in hyperspectral cubes.
  • Preprocessing: Raw HSI data underwent normalization to correct for dark noise using a dark and white reference. Bands with high noise (low and high frequencies) were removed, reducing the spectral bands from 826 to 645.
  • Band Selection: To combat the curse of dimensionality and reduce computational cost, an Ant Colony Optimization (ACO)-based band selection algorithm was employed. This algorithm aims to select a subset of bands that retain discriminative features and maximize classification accuracy, inspired by the foraging behavior of ants.
  • 3D-2D Hybrid CNN: The proposed network architecture consists of four 3D convolutional layers (to extract spectral information) followed by two 2D convolutional layers (to learn spatial features). This is followed by fully connected layers and a softmax classifier for the final four-class classification (Normal Tissue, Tumor Tissue, Blood Vessels, Background). This hybrid approach aims to overcome the limitations of purely 2D CNNs (poor spectral feature learning) and purely 3D CNNs (high complexity and potential for blurring spectral information).
  • Comparison: The proposed method was benchmarked against several other approaches: a 2D CNN, a 1D DNN, a traditional Support Vector Machine (SVM) classifier, and a hybrid approach combining the 3D-2D CNN feature extractor with an SVM classifier.

Datasets

The study utilized an in vivo human-brain HS dataset collected as part of the European project HELICoiD. This dataset consists of twenty-six HS images from sixteen adult patients. Nine of these patients had histopathologically confirmed Grade IV glioblastoma (GBM), while the remaining seven had other pathologies. The dataset was publicly available. The authors performed an inter-patient classification using Leave-One-Patient-Out Cross-Validation (LOPOCV) to ensure robust evaluation.

Key Insights and Ideas

  • Joint Spectral-Spatial Learning is Crucial: The paper strongly emphasizes that effectively combining spectral and spatial information is key to accurate HSI-based tissue classification, especially for complex biological tissues like brain tumors.
  • ACO for Band Selection is Effective: The ACO algorithm successfully identified a subset of spectral bands that significantly reduced dimensionality while preserving critical discriminative features, leading to improved classification performance and reduced computational load.
  • Hybrid CNN Architecture Outperforms Simpler Models: The 3D-2D hybrid CNN demonstrated superior performance over 2D CNNs and traditional SVMs, highlighting the benefit of its architecture in capturing both spectral and spatial nuances.
  • Tumor vs. Blood Vessels Misclassification is a Challenge: The study acknowledges that differentiating tumor tissue from blood vessels remains a significant challenge, leading to some false positives. This is attributed to the high metabolism and vascularization of tumors.
  • Specificity is High for Non-Tumor Tissue: A critical finding is the high specificity (over 97%) for non-tumor tissue (normal, blood vessels, background). This is vital for surgical applications, as it indicates a low risk of mistakenly resecting healthy tissue.

Key Results

The proposed 3D-2D hybrid CNN achieved an overall accuracy of 80% for classifying normal tissue, tumor tissue, blood vessels, and background. This represents an 8% improvement over the 2D CNN-based model.

  • Sensitivity: The model achieved a sensitivity of 76% for normal tissue, 68% for tumor tissue, 74% for blood vessels, and 96% for background.
  • Specificity: Specificity was reported as 87% for normal tissue, 98% for tumor tissue, 92% for blood vessels, and 87% for background.
  • AUC: The Area Under the Curve (AUC) for the tumor tissue was higher with the 3D-2D hybrid CNN compared to other methods.
  • Comparison: The 3D-2D hybrid CNN outperformed the 2D CNN, 1D DNN, and SVM in most metrics, particularly in accuracy and tumor sensitivity. The hybrid CNN combined with SVM showed comparable results to the standalone hybrid CNN.

Limitations and Future Work

  • Ground Truth Bias: The study acknowledges that the gold standard map (ground truth) was based on spectral similarities within the HS cubes, which could introduce a bias as these same similarities are exploited by the classification methods.
  • Limited Dense Ground Truth: The absence of a dense ground truth and clinical evaluations on tumor boundaries is noted as a limitation.
  • Non-Uniform Data Distribution: Variability in data distribution across patients and clinical centers can affect generalization. The authors suggest that a negative mask in the background could help mitigate this.
  • Influence of Resection and Blood: The study suggests that the spectral signatures might be affected by the resection process and the presence of blood, potentially leading to misclassifications.
  • Future Work: The authors propose further research into:
  • Evaluating the influence of resection on spectral signatures.
  • Developing snapshot HSI acquisition systems for real-time data capture.
  • Optimizing deep learning algorithms and multi-class classification.
  • Conducting larger patient studies and clinical evaluations to further validate the system's impact on surgical outcomes.
  • Investigating the combination of HSI with spectroscopy for better analysis of biological differences.

Summary updated

BibTeX
@inproceedings{manni2020hyperspectral,
  author = {Francesca Manni and Fons van der Sommen and Himar Fabelo and Svitlana Zinger and Caifeng Shan and Erik Edström and Adrian Elmi Terander and Samuel Ortega and Gustavo Marrero Callicó and Peter H. N. de With},
  title = {{Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach}},
  booktitle = {Sensors},
  year = {2020},
  url = {https://openreview.net/forum?id=2Izsfsn43xa},
}
Figure