TL;DR
MedViTV2 is a novel hybrid vision architecture that integrates Kolmogorov-Arnold Networks (KANs) and Dilated Neighborhood Attention (DiNA) into Transformers for robust medical image classification. It achieves state-of-the-art performance across 27 out of 29 benchmarks while being 44% more computationally efficient than its predecessor, effectively solving the "feature collapse" problem that plagues scaled-up hybrid models.
What Problem Does This Paper Address?
Computer-aided diagnosis (CAD) systems powered by deep learning have made massive strides in medical image analysis. However, standard architectures (like CNNs, Vision Transformers, and Mamba variants) are typically designed and validated on clean, curated datasets.
Real-world clinical settings are rarely that pristine. Multi-center clinical studies frequently suffer from image corruptions, noise, and artifacts arising from variations in imaging equipment across different manufacturers. Furthermore, earlier hybrid models like MedViTV1 suffer from poor scalability: as you increase the model size and parameter count, accuracy often drops due to feature collapse—a phenomenon where feature maps in deeper layers become highly redundant or entirely inactive.
The authors set out to design a generalized, robust, and highly efficient architecture capable of maintaining top-tier performance even when faced with severe, real-world image corruptions.
Methods
The authors introduce MedViTV2, a hierarchical hybrid network that balances local and global feature learning across four distinct stages. Its architecture features three primary innovations:
- KAN-Integrated Transformers: For the first time in medical vision, Kolmogorov-Arnold Network (KAN) layers replace traditional Multi-Layer Perceptrons (MLPs) in the feed-forward pathway of the Global Feature Perception (GFP) blocks. KANs learn activation functions directly on the edges (weights) rather than applying fixed activations to linear combinations. MedViTV2 utilizes an optimized variant called Reflectional Switch Activation Functions (RSWAFs) to capture complex non-linear structures efficiently.
- Dilated Neighborhood Attention (DiNA): Implemented within the Local Feature Perception (LFP) blocks, DiNA localizes self-attention to a pixel's nearest neighbors using a sliding window pattern. This grants linear time and space complexity while allowing the receptive field to expand exponentially, ensuring feature diversity and scaling stability.
- Hierarchical Hybrid Strategy (HHS): Instead of simply stacking more blocks linearly, the authors designed a specific alternating pattern (
(LFP × N + GFP × 1) × L) across the network stages to find the mathematical sweet spot between local feature extraction and global context awareness.
Datasets
The architecture was rigorously evaluated across a massive suite of 29 experimental setups:
- MedMNIST2D: 12 standardized datasets (e.g., PathMNIST, ChestMNIST, OCTMNIST) spanning 9 different imaging modalities and thousands of samples.
- NonMNIST Datasets: 5 multi-center, real-world clinical datasets including PAD-UFES-20 (skin lesions via smartphones), CPN X-ray (COVID-19/pneumonia), Fetal-Planes-DB (ultrasound), Kvasir (endoscopy), and ISIC2018.
- MedMNIST-C: A specialized benchmark extending MedMNIST to evaluate model robustness against 12 distinct types of simulated real-world image corruptions (e.g., blur, noise, brightness adjustments) across 5 severity levels.
Key Insights and Ideas
- The Power of KANs over MLPs: Traditional MLPs struggle to separate true anatomical signals from background noise in small medical datasets. Because KANs decompose functions into learnable non-linear curves, they act as superior feature extractors that require fewer parameters to capture complex medical structures.
- Mitigating Feature Collapse: Through feature cosine distance analysis, the authors proved that standard convolutional attention mechanisms tend to homogenize features in deep layers. Replacing them with sparse global attention (DiNA) combined with Local Feed-Forward Networks (LFFN) successfully preserves feature diversity during scaling.
- Structured Stacking Beats Raw Scaling: Simply increasing the number of blocks in a single stage yields diminishing returns and rapid saturation. True performance gains come from an iterative, principled combination of low-frequency (LFP) and high-frequency (GFP) signal extractors.
Key Results
- SOTA Dominance: MedViTV2 achieved state-of-the-art results in 27 out of 29 evaluation benchmarks.
- Massive Efficiency Gains: The architecture is 44% more computationally efficient than MedViTV1. As shown in the accuracy-to-FLOPs trade-off graph, MedViTV2 variants deliver significantly higher accuracy at a fraction of the computational cost compared to baselines like ResNet and MedMamba.
- Robustness Under Corruption: On the corrupted MedMNIST-C benchmark, MedViTV2 achieved a striking 13.4% accuracy improvement over previous methods, demonstrating its readiness for messy, real-world clinical deployment. It also yielded improvements of 4.6% on MedMNIST and 5.8% on NonMNIST datasets.
Limitations and Future Work
Despite its strong performance against digital and noise corruptions, the authors noted that color corruptions still cause a noticeable performance drop for MedViTV2 (a vulnerability shared with many Vision Transformer architectures).
As a direction for future work, the team aims to design specialized architectural blocks to counteract color-based distribution shifts. They also plan to adapt the MedViTV2 backbone for more complex downstream tasks, specifically 3D medical imaging and medical image segmentation.
Summary updated