TL;DR
This paper introduces UniUSNet, a novel framework that unifies ultrasound image segmentation and disease classification across diverse anatomical positions and image types. By leveraging a promptable transformer architecture and a comprehensive new dataset (BroadUS-9.7K), UniUSNet achieves state-of-the-art performance and demonstrates remarkable adaptability to new tasks with minimal fine-tuning.
What Problem Does This Paper Address?
Ultrasound imaging is a cornerstone of modern diagnostics due to its accessibility and safety. However, current AI approaches for ultrasound analysis often operate in silos, focusing on either segmentation or classification for specific organs or conditions. This leads to models that are brittle, requiring extensive retraining for new tasks or datasets, and fail to capture the full diagnostic potential of ultrasound. The field lacks a universal, adaptable framework that can handle the inherent variability of ultrasound images (different anatomical positions, image qualities) and perform multiple diagnostic tasks simultaneously.
Methods
UniUSNet is built upon a Swin-Unet-like encoder-decoder architecture, enhanced with a novel promptable learning mechanism. The core innovation lies in using four types of "prompts" – representing image nature (organ/tumor), anatomical position, diagnostic task (segmentation/classification), and input type (whole/local image) – to guide the model. These prompts are embedded and injected into each transformer layer, allowing the model to dynamically adapt its processing for different scenarios without needing to redesign the network architecture. This prompt-based approach enables zero-shot and few-shot learning capabilities, making the model highly generalizable. The framework also incorporates curriculum learning, prioritizing segmentation tasks before classification during training to leverage spatial information.
Datasets
The paper introduces BroadUS-9.7K, a new, comprehensive dataset comprising over 9.7K annotations from approximately 6.9K ultrasound images across 7 distinct anatomical positions. This dataset consolidates several public datasets (BUSI, BUSIS, UDIAT, BUS-BRA, Fatty-Liver, kidneyUS, DDTI, Fetal HC, CAMUS, Appendix) and provides annotations for both segmentation and classification tasks. This unified dataset is crucial for training and validating a universal model.
Key Insights and Ideas
- The Power of Prompts: The study strongly suggests that explicit prompts are key to achieving universal adaptability in medical imaging AI. By encoding task-specific and domain-specific information into prompts, the model can generalize far beyond its training data.
- Unified Segmentation and Classification: UniUSNet demonstrates that it's feasible and effective to train a single model for both segmenting anatomical structures and classifying diseases, rather than relying on separate models.
- BroadUS-9.7K as a Benchmark: The creation of this diverse, multi-task, multi-position dataset is a significant contribution, providing a much-needed benchmark for evaluating universal ultrasound AI models.
- Efficiency in Adaptation: The prompt-based approach, combined with adapter-based fine-tuning, allows for efficient adaptation to new tasks with minimal computational cost and without catastrophic forgetting.
Key Results
- UniUSNet achieves state-of-the-art performance on the BroadUS-9.7K dataset, outperforming single-dataset models and even SAM variants specifically adapted for ultrasound (SAMUS).
- In segmentation tasks, UniUSNet achieved an average score of 80.01%, comparable to SAMUS (80.65%) but with significantly fewer parameters.
- For classification tasks, UniUSNet showed strong performance, with an average of 79.67%.
- The model demonstrated excellent generalization capabilities, with adapter-based fine-tuning outperforming scratch training on new datasets like BUSI.
- Zero-shot inference results showed that UniUSNet significantly outperforms SAM's official weights on medical images, highlighting the importance of domain-specific adaptation.
Limitations and Future Work
- Dataset Expansion: The authors plan to further expand the BroadUS-9.7K dataset to cover more anatomical positions and rare diseases.
- Prompting Mechanism Refinement: Future work will focus on refining the prompting mechanism to enhance its expressiveness and efficiency.
- Real-World Clinical Validation: While promising, the model's performance in real-world clinical settings needs further validation.
- Computational Cost: Although more efficient than retraining, the prompt embedding and projection layers do add to the model's parameter count compared to a truly "prompt-less" model.
Summary updated