Contrastive Rendering for Ultrasound Image Segmentation

Haoming Li, Xin Yang, Jiamin Liang, Wenlong Shi, Chaoyu Chen, Haoran Dou, Rui Li, Rui Gao, Guangquan Zhou, Jinghui Fang, Xiaowen Liang, Ruobing Huang, Alejandro F. Frangi, Zhiyi Chen, Dong Ni

Abstract

Ultrasound (US) image segmentation has seen significant improvements in the deep learning era. However, the lack of sharp boundaries in US images remains an inherent challenge. Previous methods often resort to global context, multi-scale cues, or auxiliary guidance to estimate boundaries, but struggle with pixel-level learning for fine-grained boundary generation. In this paper, we propose a novel framework to improve boundary estimation. Our work has three highlights: first, we formulate boundary estimation as a rendering task to recognize ambiguous points and calibrate predictions; second, we introduce point-wise contrastive learning to enhance feature similarity within classes and decrease it between classes; third, both tasks contribute to consistent improvement while reducing network parameters. Validated on a challenging dataset of 86 ovarian US volumes, our method outperforms state-of-the-art approaches.

Addressing the Challenge of Blurry Boundaries

Ultrasound imaging is a clinical staple, yet it is notoriously difficult for automated segmentation due to low contrast, speckle noise, and shadow-occluded regions where boundaries vanish. In 3D ovarian ultrasound, follicles are small, irregular, and often appear to "touch" one another. Standard deep learning models like U-Net often produce fuzzy predictions that fail to resolve these fine-grained boundaries.

Methodology: The C-Rend Framework

The proposed Contrastive Rendering (C-Rend) framework treats boundary refinement as a rendering task rather than a simple classification problem. It introduces three main innovations to sharpen segmentation outputs:

  1. Asymmetric Backbone: A modified 3D U-Net that discards the final deconvolution layer to save computation, instead generating coarse probability maps and fine-grained feature maps from lower levels.
  2. Rendering Head: Inspired by PointRend, the model selects "ambiguous points" where uncertainty is high. These points' hybrid features are passed through a lightweight Multi-Layer Perceptron (MLP) to re-predict labels with higher precision.
  3. Point-wise Contrastive Learning: A contrastive loss forces feature representations of points from the same class to be similar while pushing different classes apart in the feature space. This specifically targets uncertain points to resolve boundary ambiguity.
The C-Rend framework architecture featuring an asymmetric backbone, rendering head, and contrastive learning module.

Experimental Results and Insights

The framework was validated on 86 3D ovarian US volumes manually annotated by expert sonographers. C-Rend achieved a Dice Similarity Coefficient (DSC) of 83.49% for follicles and 87.78% for the ovary, outperforming the baseline U-Net.

Visual comparison of ultrasound input images (a-c) and the resulting segmentation heatmap (d) highlighting the model's ability to resolve follicle boundaries.

Key findings include:

  • Efficiency: C-Rend uses fewer parameters (23.55M) than a standard U-Net (23.56M) while delivering superior results.
  • Small Object Detection: For follicles with a radius < 5mm, C-Rend reduced False Detections by 30% and Missed Detections by 15%.
  • Boundary Quality: Visualization of the Hausdorff distance confirmed that C-Rend produces cleaner, more anatomically accurate surfaces compared to edge-weighted models.

Conclusion

C-Rend demonstrates that boundary estimation in medical imaging can be significantly improved by focusing computational resources on ambiguous points through rendering and contrastive learning. While further validation on larger, more diverse datasets is needed, the framework offers a generalizable strategy for improving segmentation across various medical imaging modalities.

Summary updated

Abstract

Ultrasound (US) image segmentation embraced its significant improvement in deep learning era. However, the lack of sharp boundaries in US images still remains an inherent challenge for segmentation. Previous methods often resort to global context, multi-scale cues or auxiliary guidance to estimate the boundaries. It is hard for these methods to approach pixel-level learning for fine-grained boundary generating. In this paper, we propose a novel and effective framework to improve boundary estimation in US images. Our work has three highlights. First, we propose to formulate the boundary estimation as a rendering task, which can recognize ambiguous points (pixels/voxels) and calibrate the boundary prediction via enriched feature representation learning. Second, we introduce point-wise contrastive learning to enhance the similarity of points from the same class and contrastively decrease the similarity of points from different classes. Boundary ambiguities are therefore further addressed. Third, both rendering and contrastive learning tasks contribute to consistent improvement while reducing network parameters. As a proof-of-concept, we performed validation experiments on a challenging dataset of 86 ovarian US volumes. Results show that our proposed method outperforms state-of-the-art methods and has the potential to be used in clinical practice.
BibTeX
@misc{li2020contrastive,
  author = {Haoming Li and Xin Yang and Jiamin Liang and Wenlong Shi and Chaoyu Chen and Haoran Dou and Rui Li and Rui Gao and Guangquan Zhou and Jinghui Fang and Xiaowen Liang and Ruobing Huang and Alejandro F. Frangi and Zhiyi Chen and Dong Ni},
  title = {{Contrastive Rendering for Ultrasound Image Segmentation}},
  year = {2020},
  eprint = {2010.04928},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.04928},
}
Figure