Abstract
MISSU is a novel hybrid 3D medical image segmentation framework that synergizes the local feature extraction capabilities of Convolutional Neural Networks (CNNs) with the global modeling strengths of Transformers. Central to its design is a self-distillation mechanism that enables the encoder to learn multi-scale local details during training, which are then retained implicitly during inference. This approach allows MISSU to achieve state-of-the-art performance on benchmarks like BraTS 2019 and CHAOS while maintaining high computational efficiency.
Introduction
Precise segmentation in 3D medical imaging, such as MRI or CT scans, requires balancing local spatial details with global anatomical context. Traditional 3D U-Nets excel at local texture capture but suffer from a limited receptive field. Conversely, Transformers offer superior long-range dependency modeling but are computationally intensive for high-resolution 3D data and often lose fine-grained spatial details. Existing multi-path hybrid models often face efficiency bottlenecks, making them impractical for clinical deployment.
MISSU addresses these challenges by proposing a unified framework that models both global context and local detail during training while remaining lightweight during inference.
Methodology
The MISSU architecture follows an encoder-decoder structure integrated with a Transformer bottleneck and a self-distillation path.
Hybrid Encoder and Transformer Block
The model utilizes a 3D CNN encoder to extract hierarchical local features. These features are subsequently tokenized and processed by a Transformer block to capture global, long-range correlations across the 3D volume. This hybrid approach ensures that both fine-grained textures and broad structural relationships are preserved.
Multi-Scale Fusion and Self-Distillation
The "secret sauce" of MISSU is the Multi-Scale Fusion (MSF) block. Acting as a teacher module, it employs parallel atrous convolutions with varying dilation rates to capture rich, multi-scale information, particularly around complex organ boundaries. During training, a distillation loss ($L_{sd}$) encourages the main CNN encoder to mimic these high-quality features.
Crucially, the MSF block is discarded during inference. The knowledge it provides is distilled into the encoder's weights, allowing the model to maintain high accuracy without the added computational overhead of the teacher module at runtime.
Experiments and Results
MISSU was evaluated on the BraTS 2019 (brain tumor) and CHAOS (liver) datasets, demonstrating superior performance across multiple metrics.
Performance Benchmarks
On BraTS 2019, MISSU achieved state-of-the-art Dice scores of 89.98% for the Whole Tumor (WT), 85.77% for the Tumor Core (TC), and 80.14% for the Enhanced Tumor (ET). On the CHAOS dataset, it reached a 96.78% Dice score for liver segmentation.
Efficiency and Qualitative Analysis
Despite its performance, MISSU is highly efficient, utilizing only 10.50M parameters and 132.11 GFLOPs—significantly lower than standard 3D UNets or competing TransUNet variants. Qualitative results demonstrate MISSU's ability to accurately delineate irregular lesion boundaries compared to other methods.
Conclusion
MISSU demonstrates that implicit feature fusion via self-distillation is a powerful strategy for 3D medical image segmentation. By shifting architectural complexity to the training phase, the model achieves a superior balance between accuracy and efficiency. Future work may involve extending this framework to other imaging modalities and integrating non-image clinical metadata to further enhance diagnostic precision.
Summary updated