DeSamba: Decoupled Spectral Adaptive Framework for 3D Multi-Sequence MRI Lesion Classification

Dezhen Wang, Sheng Miao, Rongxin Chai, Jiufa CuiarXiv 2025 · arXiv · OpenReview

TL;DR

This paper introduces DeSamba, a novel framework for classifying lesions in 3D multi-sequence MRI scans. DeSamba effectively integrates information from different MRI sequences by decoupling and adaptively fusing spatial and spectral features, leading to state-of-the-art performance on challenging spinal metastasis and spondylitis datasets.

What Problem Does This Paper Address?

Classifying lesions in 3D medical images, particularly from Magnetic Resonance Imaging (MRI), is crucial for accurate diagnosis and treatment planning. MRI provides rich spatial and frequency domain information, but effectively leveraging data from multiple MRI sequences (like T1, T2, and T2-FS) for robust 3D lesion classification remains a significant challenge. Existing methods often struggle to fully exploit the discriminative power of multi-sequence data, either by treating sequences independently, simply concatenating them, or overlooking the fine-grained spectral features inherent in MRI. This can lead to missed diagnoses or inaccurate classifications, impacting patient care.

Methods

DeSamba tackles this challenge with a novel architecture comprising three main modules:

  • Multi-sequence Image Encoder: Each MRI sequence (T1, T2, T2-FS) is processed by a dual-branch encoder. One branch uses SAMNet (Spectral Adaptive Network), which incorporates a Spectral Adaptive Modulation Block (SAMB). The SAMB is the core innovation here; it transforms MRI data into the frequency domain to extract spectral features and adaptively modulates them based on lesion characteristics. This allows it to capture unique spectral patterns associated with different lesion types (e.g., osteolytic vs. osteoblastic lesions). The other branch uses a 3D MambaOut architecture for efficient sequence modeling.
  • Tabular Feature Encoder: This module processes clinical features, though its specific role and architecture are less detailed in the main text.
  • Decoupled Representation Learning Module (DRLM): This module is key to integrating information across sequences. It takes the features extracted by SAMNet and MambaOut from each sequence and decouples them into "unique" and "shared" components. This decoupling is achieved through self-reconstruction (reconstructing a sequence's features using its own unique and shared components) and cross-reconstruction (reconstructing features using components from different sequences). This process enhances sequence-specific information while also leveraging shared cues, leading to more precise modeling.

Datasets

The authors evaluated DeSamba on two clinically relevant 3D datasets:

  • Spinal Metastasis Dataset: A six-class dataset with 1,448 scans. This dataset was used for training and internal validation.
  • Spondylitis Dataset: A binary classification dataset with 251 scans. This dataset was also used for training and internal validation.

Both datasets include data from independent centers, which were used as external validation sets to assess the model's generalization capabilities.

Key Insights and Ideas

  • Spectral Signatures of Lesions: Different types of lesions (e.g., osteolytic, osteoblastic, vascularized) exhibit distinct spectral patterns in MRI. DeSamba's SAMB module is designed to capture these subtle spectral differences, which are often overlooked by purely spatial methods.
  • Decoupling for Enhanced Representation: By decoupling features into "unique" and "shared" components across different MRI sequences, DeSamba can better isolate modality-specific information while still learning complementary patterns. This is crucial for multi-sequence analysis where sequences offer both unique and overlapping information.
  • Adaptive Fusion is Key: The framework doesn't just combine features; it adaptively fuses them based on lesion characteristics. This dynamic approach allows the model to prioritize relevant spectral and spatial information for accurate classification.
  • Mamba for Long-Range Dependencies: The inclusion of MambaOut in the architecture suggests an exploration of efficient sequence modeling for potentially long volumetric data, a growing trend in medical imaging.

Key Results

DeSamba demonstrated state-of-the-art performance on both datasets:

  • Spinal Metastasis: Achieved 62.10% Top-1 accuracy and 87.71% AUC on the external validation set, outperforming all baseline models. It also reached 93.55% Top-3 accuracy.
  • Spondylitis: Achieved 64.52% accuracy and 73.88 AUC on the external validation set.
  • Ablation Studies: Both the Decoupled Representation Learning Module (DRLM) and the Spectral Adaptive Modulation Block (SAMB) significantly contributed to the model's performance, with over 10% relative improvement compared to baselines.
  • Efficiency: DeSamba offers a favorable balance between performance and computational efficiency, achieving high accuracy with computational costs comparable to Transformer and SSM-based models.

Limitations and Future Work

  • Dataset Limitations: The study relies on two private datasets. While external validation was performed, the availability of larger, publicly accessible multi-sequence 3D MRI datasets would further validate the model's generalizability.
  • Tabular Encoder Details: The specific architecture and contribution of the tabular feature encoder are less elaborated upon in the main text.
  • Future Work: The authors highlight DeSamba's potential for robust, generalizable multi-sequence medical image analysis in clinical practice. Future work could involve exploring its application to other types of 3D medical imaging tasks and datasets.

Summary updated

Abstract

Magnetic Resonance Imaging (MRI) sequences provide rich spatial and frequency domain information, which is crucial for accurate lesion classification in medical imaging. However, effectively integrating multi-sequence MRI data for robust 3D lesion classification remains a challenge. In this paper, we propose DeSamba (Decoupled Spectral Adaptive Network and Mamba-Based Model), a novel framework designed to extract decoupled representations and adaptively fuse spatial and spectral features for lesion classification. DeSamba introduces a Decoupled Representation Learning Module (DRLM) that decouples features from different MRI sequences through self-reconstruction and cross-reconstruction, and a Spectral Adaptive Modulation Block (SAMB) within the proposed SAMNet, enabling dynamic fusion of spectral and spatial information based on lesion characteristics. We evaluate DeSamba on two clinically relevant 3D datasets. On a six-class spinal metastasis dataset (n=1,448), DeSamba achieves 62.10% Top-1 accuracy, 63.62% F1-score, 87.71% AUC, and 93.55% Top-3 accuracy on an external validation set (n=372), outperforming all state-of-the-art (SOTA) baselines. On a spondylitis dataset (n=251) involving a challenging binary classification task, DeSamba achieves 70.00%/64.52% accuracy and 74.75/73.88 AUC on internal and external validation sets, respectively. Ablation studies demonstrate that both DRLM and SAMB significantly contribute to overall performance, with over 10% relative improvement compared to the baseline. Our results highlight the potential of DeSamba as a generalizable and effective solution for 3D lesion classification in multi-sequence medical imaging.
BibTeX
@misc{wang2025desamba,
  author = {Dezhen Wang and Sheng Miao and Rongxin Chai and Jiufa Cui},
  title = {{DeSamba: Decoupled Spectral Adaptive Framework for 3D Multi-Sequence MRI Lesion Classification}},
  year = {2025},
  eprint = {2507.15487},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.15487},
}
Figure