Diagnostic Performance of Universal-Learning Ultrasound AI Across Multiple Organs and Tasks: the UUSIC25 Challenge

Zehui Lin, Luyi Han, Xin Wang, Ying Zhou, Yanning Zhang, Tianyu Zhang, Lingyun Bao, Shandong Wu, Dong Xu, Tao Tan, Uusic25 Challenge ConsortiumarXiv 2025 · DOI · arXiv

Introduction

Modern ultrasound hardware is inherently universal; a single probe can be used to image the heart, liver, or a developing fetus. However, ultrasound artificial intelligence (AI) has historically remained fragmented, following a "one-organ-one-task" paradigm. This fragmentation forces clinical centers to deploy and maintain multiple specialized models, creating a significant utility gap in clinical workflows. The Universal UltraSound Image Challenge 2025 (UUSIC25) was organized to address this by evaluating "Generalist AI" capable of simultaneous segmentation and classification across diverse anatomical regions without compromising accuracy or computational speed.

Data and Evaluation Framework

The challenge utilized a massive, heterogeneous dataset comprising 16,021 images across seven anatomical regions: breast, thyroid, liver, kidney, fetal head, cardiac, and appendix. The data was structured to test both internal consistency and real-world generalization.

The UUSIC25 cohort includes 16,021 images from public and private sources, including a strictly external held-out test set from the Netherlands.

As shown in the cohort flow, the training set aggregated nine public datasets with a 30% "calibration" slice of private data. Crucially, the test set included a strictly external cohort from the Netherlands Cancer Institute (NKI), which remained entirely unseen during the training phase to provide a rigorous assessment of domain generalization.

The challenge evaluated 15 valid algorithms using a multi-dimensional ranking system that weighted diagnostic performance at 70% and computational efficiency (inference time and GPU memory) at 30%.

Analysis of the top-performing models revealed several key trends:

  • Architecture: There was a decisive shift toward Transformer-based backbones over traditional CNNs, allowing models to better capture long-range contextual information across different organ types.
  • Unified Pipelines: Leading entries utilized "implicit queries" or "tags" within a single network to dynamically focus the model on specific organs or tasks.
  • Efficiency Constraints: To discourage computationally expensive "ensemble bloat," the ranking penalized high memory usage, favoring models optimized for portable, point-of-care devices.

Results and the Efficiency Frontier

The results demonstrated that versatility does not necessarily preclude precision. The winning model, SMART, achieved a macro-averaged Dice Similarity Coefficient (DSC) of 0.854 for segmentation and a macro-averaged AUC of 0.766 for binary classification.

Multi-panel analysis of model architectures, organ-specific performance (radar charts), and the trade-off between performance and efficiency.

As illustrated in the performance vs. efficiency analysis, the SMART model occupied the "Balanced Zone," requiring only 0.59 GB of VRAM. This represents a 94% reduction in overhead compared to running seven separate specialized models. While segmentation tasks showed high performance (e.g., 0.942 DSC for fetal heads), classification tasks—particularly those involving sub-visual biomarkers like breast cancer molecular subtyping—proved significantly more challenging.

Discussion and Future Directions

While UUSIC25 proves the feasibility of generalist ultrasound AI, several hurdles remain. The "Generalization Gap" was evident in the performance drop on data from the external NKI center, suggesting that domain generalization techniques must be a priority for future research. Furthermore, the current challenge was limited to static B-mode images; future iterations must incorporate video/cine loops and multi-modal data (e.g., Color Doppler) to better reflect real-time clinical scanning.

Finally, the shift toward "all-in-one" models presents new regulatory challenges. Current frameworks are largely designed for single-indication devices, and validating a single model that handles seven organs simultaneously will require the development of new, robust validation standards.

Summary updated

BibTeX
@misc{lin2025diagnostic,
  author = {Zehui Lin and Luyi Han and Xin Wang and Ying Zhou and Yanning Zhang and Tianyu Zhang and Lingyun Bao and Shandong Wu and Dong Xu and Tao Tan and UUSIC25 Challenge Consortium},
  title = {{Diagnostic Performance of Universal-Learning Ultrasound AI Across Multiple Organs and Tasks: the UUSIC25 Challenge}},
  year = {2025},
  eprint = {2512.17279},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.17279},
}
Figure