PH-Net: Semi-Supervised Breast Lesion Segmentation via Patch-Wise Hardness

Siyao Jiang, Huisi Wu, Junyang Chen, Qin Zhang, Jing QinCVPR 2024 · DOI

Abstract

Accurate segmentation of breast lesions in ultrasound (BUS) images is critical for early cancer detection but is hindered by speckle noise, ambiguous boundaries, and the scarcity of labeled data. We present PH-Net, a semi-supervised learning framework that prioritizes "hard" image patches to improve segmentation performance. PH-Net utilizes an entropy-based hardness metric to identify challenging regions, applying Adaptive Patch Augmentation (APA) to preserve these areas and Hard-Patch Contrastive Learning (HPC) to enhance feature discrimination. Experimental results on the UDIAT and BUSI datasets show that PH-Net consistently outperforms existing state-of-the-art methods, especially when labeled data is extremely limited.

Introduction

Breast ultrasound imaging is a primary screening tool, yet automated lesion segmentation remains a significant challenge. The difficulty stems from two main factors: the inherent complexity of ultrasound images—characterized by speckle noise and artifacts—and the high cost of obtaining pixel-level manual annotations from medical experts.

While semi-supervised learning offers a path forward by leveraging unlabeled data, traditional methods often fail to distinguish between "easy" and "hard" regions within an image. This lack of differentiation can lead to models that perform well on clear lesion interiors but struggle with ambiguous boundaries. PH-Net addresses this by focusing specifically on difficult patches, ensuring the model learns effectively from the most informative parts of the data.

The PH-Net Framework

PH-Net is built upon a Teacher-Student architecture designed to maximize the utility of unlabeled images through targeted learning strategies.

Overview of the PH-Net architecture featuring Teacher-Student branches and Adaptive Patch Augmentation.

Hardness Metric and Adaptive Augmentation

The core of PH-Net is its ability to quantify patch-wise difficulty. The model divides images into small patches and calculates a "hardness score" based on average entropy. High entropy indicates areas where the model is uncertain, typically corresponding to noisy regions or lesion boundaries.

To prevent these critical areas from being lost during data augmentation, PH-Net introduces Adaptive Patch Augmentation (APA). Unlike standard methods like CutMix that randomly replace image sections, APA identifies the top-$k$ hardest patches and "shields" them. This ensures that the student model is consistently exposed to the most challenging anatomical features during training.

Hard-Patch Contrastive Learning (HPC)

To further refine the model's discriminative power, Hard-Patch Contrastive Learning (HPC) is applied. This technique forces the model to cluster features of the same class (e.g., lesion) while pushing apart features of different classes (e.g., background) specifically within the identified hard patches. A memory bank is utilized to store feature centers, providing a stable reference and preventing training instability.

Experimental Evaluation

The framework was evaluated on two public datasets:

  1. UDIAT Dataset: 163 images (110 benign, 53 malignant).
  2. BUSI Dataset: 647 abnormal images (benign and malignant).

The researchers simulated real-world data scarcity by testing the model under three labeling protocols: 1/2, 1/4, and 1/8 of the total data being labeled.

Results and Discussion

PH-Net demonstrated superior performance across all tested scenarios. On the UDIAT dataset with only 1/8 labeled data (approximately 15 images), PH-Net achieved a Dice score of 84.99%, surpassing the next best semi-supervised method by nearly 1%. Similar gains were observed on the BUSI dataset, where it reached a 73.61% Dice score under the 1/8 protocol, a significant improvement over the supervised-only baseline of 70.99%.

Quantitative comparison of PH-Net against state-of-the-art methods on UDIAT and BUSI datasets.

Qualitative analysis revealed that PH-Net produces more accurate lesion boundaries and is less prone to "leaking" into surrounding tissue compared to other models. This success highlights the importance of treating image regions with varying levels of difficulty differently.

Conclusion

PH-Net introduces a novel approach to semi-supervised medical image segmentation by prioritizing "hard" patches. By shielding difficult regions from aggressive augmentation and applying targeted contrastive learning, it achieves state-of-the-art results with minimal labeled data. While challenges remain in segmenting extremely ambiguous contours, PH-Net represents a significant step toward more robust and data-efficient breast cancer screening tools.

Summary updated

BibTeX
@inproceedings{jiang2024ph,
  author = {Siyao Jiang and Huisi Wu and Junyang Chen and Qin Zhang and Jing Qin},
  title = {{PH-Net: Semi-Supervised Breast Lesion Segmentation via Patch-Wise Hardness}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.01085},
}
Figure