URFM: a general Ultrasound Representation Foundation Model for advancing ultrasound image diagnosis

Qingbo Kang, Qicheng Lao, Jun Gao, Wuyongga Bao, Qiang Lu, Kang Li

Abstract

Ultrasound imaging is pivotal in clinical diagnostics, providing critical insights into a wide range of diseases and organs. However, advancing artificial intelligence (AI) in this field is hindered by challenges such as the reliance on large labeled datasets and the limited generalizability of task-specific models, largely due to ultrasound’s low signal-to-noise ratio (SNR). To address these issues, we propose the Ultrasound Representation Foundation Model (URFM), designed to learn robust and generalizable representations from unlabeled ultrasound data. URFM is pre-trained on over 1 million ultrasound images from 15 major anatomical organs, utilizing representation-based masked image modeling (MIM) and state-of-the-art self-supervised learning techniques. Unlike traditional pixel-based MIM, URFM integrates high-level representations from BiomedCLIP, a medical vision-language model, to handle the low SNR inherent in ultrasound imaging. Extensive evaluations show URFM outperforms existing methods, excelling in generalization, label efficiency, and training speed, highlighting its potential to revolutionize diagnostic accuracy and clinical workflows.

Introduction

Ultrasound is a cornerstone of modern medicine due to its non-invasive, cost-effective, and real-time imaging capabilities. Despite its ubiquity, developing robust AI models for ultrasound interpretation remains a significant challenge. Three primary obstacles hinder progress: the low signal-to-noise ratio (SNR) caused by speckle noise, the "organ barrier" where models fail to generalize across different anatomical regions, and the scarcity of high-quality, expert-labeled datasets. While foundation models have transformed natural image processing, their application to ultrasound has been limited because standard self-supervised learning methods often struggle to distinguish clinical features from imaging artifacts.

Methodology: Breaking the Noise Barrier

The Ultrasound Representation Foundation Model (URFM) addresses these challenges by shifting the focus from raw pixels to semantic features. Built on a Vision Transformer (ViT-B/16) architecture, URFM employs a novel Representation-based Masked Image Modeling (MIM) approach.

In traditional MIM frameworks, such as Masked Autoencoders (MAE), the model is tasked with reconstructing missing pixels from a partially masked image. However, in the context of ultrasound, pixel-level reconstruction often forces the model to learn noise patterns rather than anatomical structures. URFM overcomes this by reconstructing high-level feature embeddings instead. During pre-training, the model predicts the representations that would be generated by BiomedCLIP, a pre-trained medical vision-language model, for the masked portions of the image. This knowledge distillation process ensures that the model learns clinically relevant features, effectively filtering out the inherent noise of ultrasound imaging.

Figure 1: Overview of the URFM framework, including dataset construction, the representation-based MIM pre-training architecture, and downstream clinical applications.

Data and Experimental Design

To build a truly generalizable model, the researchers curated a massive pre-training dataset comprising 1,003,465 images across 15 major anatomical organs, including the thyroid, breast, liver, kidney, and heart. This dataset combines public repositories like RadImageNet and BUV with a substantial private collection from West China Hospital.

The model's performance was rigorously evaluated across 10 downstream clinical applications involving 9 different organs. These benchmarks included tasks such as thyroid nodule classification (Thyroid4K), breast tumor detection (BUS-BRA), liver mass diagnosis (Liver735), and fetal plane detection (Fetus12K).

Key Results and Performance

URFM consistently established new state-of-the-art (SOTA) benchmarks across all evaluated tasks, outperforming existing models like USFM and ImageNet-based pre-training.

  • Diagnostic Accuracy: URFM achieved a mean accuracy of 86.35% and a mean F1-score of 85.42% across the ten clinical tasks. Notably, in liver mass diagnosis, the F1-score improved from 74.89% (previous SOTA) to 79.33%.
  • Label Efficiency: One of the most significant advantages of URFM is its ability to perform well with limited data. The model achieved higher performance using only 20% of labeled data than competing models did with the full 100% dataset.
  • Generalization: By training on a diverse 15-organ dataset, URFM successfully "broke the organ barrier." Unlike task-specific models that degrade when applied to new anatomy, URFM maintains high performance across the entire spectrum of ultrasound diagnostics.
Figure 2: Quantitative performance comparison of URFM against baseline models. Radar charts show superior metrics across all tasks, while bar charts and ROC curves highlight specific gains in accuracy and reliability.

Discussion and Future Directions

The success of URFM highlights two critical insights for medical AI: first, that feature-level reconstruction is superior to pixel-level reconstruction in low-SNR environments; and second, that large-scale, multi-organ pre-training is essential for creating versatile diagnostic tools. The study also identified a "data scaling limit," suggesting that beyond a certain point (approximately 100,000 images per organ), the quality and diversity of data become more important than sheer volume.

While URFM represents a major leap forward, future work will focus on extending the model to handle temporal data from ultrasound video (cine loops) and validating its performance in real-world clinical workflows. By releasing the code and pre-trained weights, the authors aim to provide a robust foundation for the next generation of ultrasound AI applications.

Summary updated

Abstract

<title>Abstract</title> Ultrasound imaging is pivotal in clinical diagnostics, providing critical insights into a wide range of diseases and organs. However, advancing artificial intelligence (AI) in this field is hindered by challenges such as the reliance on large labeled datasets and the limited generalizability of task-specific models, largely due to ultrasound’s low signal-to-noise ratio (SNR). To address these issues, we propose the Ultrasound Representation Foundation Model (URFM), designed to learn robust and generalizable representations from unlabeled ultrasound data.URFM is pre-trained on over 1 million ultrasound images from 15 major anatomical organs, utilizing representation-based masked image modeling (MIM) and state-of-the-art self-supervised learning techniques. Unlike traditional pixel-based MIM, URFM integrates high-level representations from BiomedCLIP, a medical vision-language model, to handle the low SNR inherent in ultrasound imaging. Extensive evaluations show URFM outperforms existing methods, excelling in generalization, label efficiency, and training speed, highlighting its potential to revolutionize diagnostic accuracy and clinical workflows.
BibTeX
@inproceedings{kang2025urfm,
  author = {Qingbo Kang and Qicheng Lao and Jun Gao and Wuyongga Bao and Qiang Lu and Kang Li},
  title = {{URFM: a general Ultrasound Representation Foundation Model for advancing ultrasound image diagnosis}},
  booktitle = {iScience},
  year = {2025},
  doi = {10.1016/j.isci.2025.112917},
}
Figure