Abstract
Identifying small, highly variable brain structures like the paracingulate sulcus (PCS) remains a significant challenge in neuroimaging. This study systematically benchmarks four pre-training and four fine-tuning strategies using a Swin-UNETR architecture on the TOP-OSLO cohort. Our results indicate that combining reconstruction or adversarial pre-training with "top-level" fine-tuning—updating only the final 10% of the model—provides an optimal balance of segmentation accuracy and computational efficiency, outperforming popular methods like Low-Rank Adaptation (LoRA) in this specific medical imaging context.
Introduction
The paracingulate sulcus (PCS) is a secondary brain structure characterized by significant inter-individual variation in shape, size, and location. For computer vision models, identifying the PCS is a "needle-in-a-haystack" problem. While the field has seen a surge in new training paradigms—including diffusion models, contrastive learning, and parameter-efficient fine-tuning (PEFT)—there is no consensus on which combination is best suited for niche medical segmentation tasks. This paper fills that gap by evaluating these techniques to find the most efficient path to high-precision neuroimaging.
Methods
The researchers employed a Swin-UNETR architecture, a 3D transformer-based neural network, to test a matrix of strategies across two training phases:
Pre-training (Self-Supervised)
- Simple Reconstruction: Learning to recreate the input brain image.
- Adversarial Learning: A GAN-style setup to distinguish real from reconstructed images.
- Contrastive Learning: Grouping similar brain features together.
- Diffusion Denoising: Recovering images from added noise.
Fine-tuning (Task-Specific)
- Full Tuning: Updating the entire network.
- Decoder Tuning: Updating only the decoder portion.
- Top-Tuning: Updating only the last 10% of the model's parameters.
- LoRA (Low-Rank Adaptation): Injecting trainable low-rank matrices into linear layers.
The study utilized the TOP-OSLO cohort, comprising T1-weighted MRI scans from 596 subjects, focusing on the white-grey surface matter of the left hemisphere.
Key Results and Insights
Performance of Pre-training Strategies
Simple reconstruction and adversarial pre-training captured structural nuances better than diffusion models. Notably, diffusion denoising was significantly more computationally expensive (nearly 3,500 seconds per epoch) compared to simple reconstruction (179 seconds) without providing a performance advantage for this task.
The "Top-Tuning" Advantage
Tuning just the top 10% of the decoder parameters achieved peak performance, with classification accuracy exceeding 60% when paired with reconstruction pre-training. This strategy maintained high segmentation fidelity while significantly reducing GPU memory requirements and training time.
Limitations of LoRA in Vision
While LoRA is a standard for fine-tuning Large Language Models, it struggled in this spatial context. The high complexity of small brain regions appears to require more substantial weight updates than the low-rank approximations LoRA provides.
Conclusion
Top-tuning consistently delivered average Dice scores exceeding 50% across pre-training simulations. By hovering around 200 seconds per epoch and keeping GPU memory allocation below 92.20%, this approach makes high-precision neuroimaging more accessible for researchers with limited hardware. Future work will focus on testing these models against Out-of-Distribution (OOD) cohorts to ensure generalizability across different scanners and populations.
Summary updated