Study on an Automatic Classification Method for Determining the Malignancy Grade of Glioma Pathological Sections Based on Hyperspectral Multi-Scale Spatial-Spectral Fusion Features

Jiaqi Chen, Jin Yang, Jinyu Wang, Zitong Zhao, Mingjia Wang, Ci Sun, Nan Song, Shulong Feng2024 · DOI

TL;DR

This paper introduces SMLMER-ResNet, a novel deep learning model that leverages hyperspectral imaging and a multi-scale spatial-spectral fusion approach to automatically classify the malignancy grade of brain glioma pathological sections. The model achieves high accuracy, offering a promising tool to assist pathologists and improve diagnostic efficiency.

What Problem Does This Paper Address?

Diagnosing and grading diffuse gliomas, a common type of brain tumor, is crucial for determining treatment and prognosis. Traditionally, this relies on histopathology, which can be subjective and prone to inter-observer variability. The World Health Organization (WHO) classifies gliomas into grades I-IV based on their histological characteristics, but achieving consistent and accurate grading is challenging due to tumor heterogeneity and the nuanced nature of diagnostic criteria. While advancements in genetic analysis have improved understanding, there's a persistent need for objective, stable, and reproducible methods to grade these tumors, guiding clinical decisions more effectively.

Methods

The core of this work is the SMLMER-ResNet model, a deep learning architecture designed to process hyperspectral images (HSIs) of glioma pathology. Key methodological components include:

  • Hyperspectral Imaging: The study utilizes a custom-built micro-hyperspectral imaging system to capture detailed spectral information from pathological slices, covering the 400-1000 nm range with 270 spectral channels.
  • Multi-Scale Spatial-Spectral Feature Extraction: The model employs a two-pronged approach:
  • Spectral Feature Extraction: This module uses residual networks and spectral attention mechanisms (SE module) to learn correlations across spectral channels, fusing features and reducing redundant information. It processes image blocks of varying sizes (e.g., 7x7) to capture multi-scale spectral characteristics.
  • Spatial Feature Extraction: This module focuses on extracting spatial features from the HSI data, incorporating spatial attention mechanisms to highlight important regions. It also uses multi-scale processing and hierarchical fusion to capture both local and abstract semantic information.
  • Channel Attention Mechanism (SE Module): Integrated into the spectral feature extraction, this mechanism adaptively recalibrates channel-wise feature responses, improving feature representation.
  • Hierarchical Spatial Feature Fusion: This module aggregates spatial features from different network depths (low, mid, and high-level) to create a comprehensive representation.
  • Data Dimensionality Reduction: Techniques like Simulated Annealing Algorithm (SAA) and Successive Projections Algorithm (SPA) were explored to reduce the high dimensionality of hyperspectral data, with SAA showing superior performance.
  • Classifier: The model utilizes a classifier that combines a fully connected layer, a sigmoid activation function, and L2 regularization for robust classification.

Datasets

The study utilized a dataset of 109 brain glioma cases, spanning different grades (14 grade 1, 64 grade 2, 46 grade 3, and 72 grade 4), along with 22 normal brain tissue samples. These samples were processed using formalin fixation, HE staining, and IHC staining. The data was collected using a micro-hyperspectral imaging system.

Key Insights and Ideas

  • Hyperspectral Imaging's Power: The study highlights how hyperspectral imaging, by capturing detailed spectral signatures, offers richer information than traditional RGB images for pathological analysis, enabling the detection of subtle molecular changes.
  • Synergy of Spatial and Spectral Features: The core innovation lies in effectively fusing multi-scale spatial and spectral features. The model demonstrates that combining these distinct information sources leads to superior classification performance compared to relying on either alone.
  • Attention Mechanisms are Crucial: The inclusion of both spectral and spatial attention mechanisms significantly boosts performance, underscoring their importance in guiding the network to focus on the most informative spectral bands and spatial regions.
  • Multi-Scale Processing is Key for Gliomas: Glioma tumors exhibit heterogeneity. Processing image blocks at multiple scales allows the model to capture both fine-grained details and broader contextual information, essential for accurate grading.
  • SAA Outperforms SPA for Dimensionality Reduction: For this specific task, the Simulated Annealing Algorithm proved more effective than the Successive Projections Algorithm in reducing hyperspectral data dimensionality while preserving crucial information.

Key Results

The proposed SMLMER-ResNet model achieved impressive results:

  • High Classification Accuracy: An average classification accuracy of 97.32% was reported.
  • Strong Kappa Coefficient: A Kappa coefficient of 0.954 indicates excellent agreement and consistency in classification.
  • Optimal Input Size: The study identified optimal input image block sizes of 7x7 for the spectral feature extraction module and 27x27 for the spatial feature extraction module.
  • Learning Rate Optimization: A learning rate of 0.001 was found to yield peak classification accuracy.
  • Ablation Study Success: The ablation experiments confirmed the effectiveness of the attention mechanisms and the proposed classifier, with the full SMLMER-ResNet model outperforming configurations with individual attention mechanisms or simpler classifiers.
  • Comparison to Other Models: SMLMER-ResNet demonstrated superior performance compared to traditional machine learning algorithms (SVM, DT, RFR) and other deep learning models like SE-ResNet, CNN, 2D_CNN, and 3D_CNN.

Limitations and Future Work

  • Sample Size: The authors acknowledge that the current study has a small sample size.
  • Future Work:
    • Increase Sample Size: The plan is to expand the dataset and further validate the model using clinical resources.
  • Explore Band Importance: Future research will investigate which specific hyperspectral bands or pathological features contribute most to classification accuracy.
  • Clinical Integration: The goal is to work with hospitals to complete studies on medical mechanisms and improve the interpretability of the test, aiming for wider adoption in clinical settings.
  • Interpretability: Further exploration into the interpretability of the model's decisions is also a future direction.

Summary updated

BibTeX
@misc{chen2024study,
  author = {Jiaqi Chen and Jin Yang and Jinyu Wang and Zitong Zhao and Mingjia Wang and Ci Sun and Nan Song and Shulong Feng},
  title = {{Study on an Automatic Classification Method for Determining the Malignancy Grade of Glioma Pathological Sections Based on Hyperspectral Multi-Scale Spatial-Spectral Fusion Features}},
  year = {2024},
  doi = {10.3390/s24123803},
}
Figure