Hyperspectral imaging in neurosurgery: a review of systems, computational methods, and clinical applications

Alankar Kotwal, Vishwanath Saragadam, Joshua D. Bernstock, Alfredo Sandoval, Ashok Veeraraghavan, Pablo A. Valdés

TL;DR

Hyperspectral imaging (HSI) offers a powerful, non-invasive way to "see" beyond visible light, enabling surgeons to better distinguish between healthy and diseased brain tissue during complex neurosurgical procedures. This review details the current state of HSI in neurosurgery, highlighting its systems, computational methods, and promising clinical applications across various subspecialties, paving the way for improved surgical guidance and patient outcomes.

What Problem Does This Paper Address?

Accurate identification of pathological (e.g., tumors) versus healthy brain tissue is paramount in neurosurgery. However, current surgical adjuncts like image guidance systems can become inaccurate due to brain shift during surgery, and visual inspection alone is often insufficient to detect subtle differences, especially in infiltrative tumors like glioblastomas. This leads to challenges in maximizing tumor resection while preserving critical brain function. HSI offers a potential solution by providing a detailed spectral "fingerprint" of tissues, allowing for more precise differentiation.

Methods

This paper reviews hyperspectral imaging (HSI) techniques applied in neurosurgery. It categorizes and explains various HSI systems, including:

  • Point scanning: Acquires spectra pixel by pixel, offering high spectral resolution but slow acquisition.
  • Line scanning (Pushbroom): Captures spectral data row by row, reducing acquisition time compared to point scanning.
  • Spectral scanning: Images one spectral channel at a time using tunable filters, offering flexibility but potentially high time costs.
  • Snapshot: Captures the entire hyperspectral cube in a single exposure, enabling real-time applications but often with trade-offs in spatial/spectral resolution.
  • Snapscan: Combines benefits of snapshot and line scanning.
  • Compressed sensing: Exploits signal regularity for faster acquisition.

The review also details computational methods for processing HSI data, including classification, estimation, and band selection algorithms, as well as visualization techniques like augmented reality overlays.

Datasets

The paper discusses several publicly available datasets, primarily from the HELICoiD project, which aim to facilitate research in neurosurgical HSI. These include:

  • HELICOiD Sample In-Vivo HS Human Brain Database: Contains VIS-NIR hyperspectral cubes of glioblastoma multiforme (GBMs) with labeled pixels for normal tissue, tumor tissue, blood vessel, and background.
  • HELICOiD Full In-Vivo HS Human Brain Database: An extended version with more cubes and patients, including data on oligodendrogliomas, meningiomas, and metastases.
  • HELICOiD Enhanced In-Vivo HS Human Brain Database (Benchmark): Further data for validating classification techniques.
  • Micro-surgical hyperspectral database: A systematically designed database under development.
  • SLIM Brain Database: Contains multimodal data (hyperspectral, RGB-depth) from multiple cameras.
  • Vascular and functional neurosurgery datasets: Include hyperspectral video captured during surgery for epilepsy studies.
  • Spine surgery datasets: Data from bovine calf cadavers for spinal fusion studies.

Key Insights and Ideas

  • HSI as an "Optical Fingerprint": The core idea is that HSI captures a unique spectral signature for each tissue type, allowing for differentiation beyond what the human eye or conventional cameras can perceive.
  • Beyond Reflectance: Fluorescence Matters: While reflectance HSI is valuable for tissue classification, fluorescence HSI, particularly with agents like 5-ALA-induced PpIX, significantly enhances tumor visualization by increasing contrast, especially for infiltrative tumors.
  • The Speed-Spectral-Sensitivity Trade-off: A fundamental challenge is balancing the need for high spectral and spatial resolution with fast acquisition rates required for real-time surgical guidance. Snapshot and snapscan systems are key to addressing this.
  • Computational Power is Crucial: Processing the vast amount of data generated by HSI requires sophisticated algorithms and hardware acceleration (GPUs, FPGAs) to achieve clinically relevant speeds.
  • Data Availability is Key for Progress: The availability of well-annotated datasets is critical for training and validating machine learning models, driving advancements in HSI applications.

Key Results

  • Broad Applicability: HSI has shown promise across major neurosurgical subspecialties: neurosurgical oncology, vascular neurosurgery, functional neurosurgery, and spine surgery.
  • High Accuracy in Classification: Studies report classification accuracies exceeding 99% for tumor identification using HSI, with some algorithms achieving high sensitivity and specificity.
  • Improved Tumor Detection: HSI, especially fluorescence-guided HSI, can detect residual tumors and infiltrative margins that are often invisible with conventional methods.
  • Hemodynamic and Functional Mapping: HSI can accurately estimate cerebral oxygenation, map blood flow, and detect neurovascular coupling, aiding in functional brain mapping and seizure localization.
  • Real-time Potential: While challenges remain, snapshot and snapscan systems are enabling near-real-time acquisition and processing, crucial for intraoperative guidance.

Limitations and Future Work

Limitations:

  • Speed and Footprint: Current HSI systems can be bulky and slow, potentially interrupting surgical workflow.
  • Spatial and Spectral Resolution Trade-offs: Achieving high resolution in both dimensions simultaneously remains a challenge.
  • Inter-patient Variability: Reflectance spectra can vary between patients, impacting the generalization of trained models.
  • Limited Penetration Depth: HSI primarily images superficial tissues, limiting its ability to assess deep structures.
  • Data Scarcity and Labeling: While datasets are growing, large, diverse, and perfectly labeled datasets are still needed.

Future Work:

  • Real-time Performance: Further optimization of acquisition, processing, and projection pipelines is needed to achieve true real-time frame rates.
  • Miniaturization and Integration: Developing smaller, more integrated HSI systems that seamlessly fit into existing surgical setups.
  • Improved Computational Methods: Exploring advanced dimensionality reduction, deep learning architectures, and efficient algorithms for faster and more accurate analysis.
  • Fusion with Other Modalities: Combining HSI with MRI, ultrasound, and neuronavigation for a more comprehensive intraoperative view.
  • Standardization: Developing standardized protocols, data formats, and clinical study designs for better reproducibility and comparability.
  • Addressing Deep Tissue Imaging: Investigating techniques to extend HSI's penetration depth or integrate it with methods that can image deeper structures.

Summary updated

Abstract

We describe a number of HSI applications across every major branch of neurosurgery. We believe these results demonstrate the potential of HSI as a powerful neurosurgical adjunct as more work continues to enable rapid acquisition with smaller footprints, greater spectral and spatial resolutions, and improved detection.
BibTeX
@article{kotwal2024hyperspectral,
  author = {Alankar Kotwal and Vishwanath Saragadam and Joshua D. Bernstock and Alfredo Sandoval and Ashok Veeraraghavan and Pablo A. Valdés},
  title = {{Hyperspectral imaging in neurosurgery: a review of systems, computational methods, and clinical applications}},
  journal = {Journal of Biomedical Optics},
  year = {2024},
  doi = {10.1117/1.jbo.30.2.023512},
}
Figure