Abstract
Medical image segmentation typically necessitates a large and precisely annotated dataset. However, obtaining pixel-wise annotation is a labor-intensive task that requires significant effort from domain experts, making it challenging to obtain in practical clinical scenarios. In such situations, reducing the amount of annotation required is a more practical approach. One feasible direction is sparse annotation, which involves annotating only a few slices, and has several advantages over traditional weak annotation methods such as bounding boxes and scribbles, as it preserves exact boundaries. However, learning from sparse annotation is challenging due to the scarcity of supervision signals. To address this issue, we propose a framework that can robustly learn from sparse annotation using the cross-teaching of both 3D and 2D networks. Considering the characteristic of these networks, we develop two pseudo label selection strategies, which are hard-soft confidence threshold and consistent label fusion. Our experimental results on the MMWHS dataset demonstrate that our method outperforms the state-of-the-art (SOTA) semi-supervised segmentation methods. Moreover, our approach achieves results that are comparable to the fully-supervised upper bound result.
Introduction
Deep learning has revolutionized medical imaging, but its success is often predicated on the availability of massive, densely annotated datasets. For 3D volumetric tasks, such as heart or organ segmentation, a single patient volume can contain hundreds of slices. Requiring radiologists to provide pixel-perfect labels for every slice is economically and practically unfeasible in many clinical settings.
While weak annotations like bounding boxes or scribbles offer a faster alternative, they often fail to capture the precise anatomical boundaries necessary for surgical planning or diagnostic accuracy. Sparse annotation—labeling only a small subset of representative slices—presents a compelling middle ground. However, the resulting "thin" supervision signal makes it difficult for standard models to generalize across the unlabeled gaps. This paper introduces a 3D-2D Cross-Teaching framework designed to overcome these limitations by leveraging the complementary strengths of different network architectures.
Methodology
The proposed framework utilizes a co-training principle where multiple models provide pseudo-labels for one another on unlabeled data. The core innovation lies in the 3D-2D Cross-Teaching architecture, which employs one 3D network (V-Net) and two 2D networks (U-Nets).
Cross Annotation Strategy
To maximize anatomical diversity with minimal effort, the researchers utilize a "cross annotation" strategy. Instead of labeling random slices, they label slices from two orthogonal planes (transverse and coronal). This creates a cross-shaped sampling in 3D space, ensuring the model receives supervision across different orientations.
Pseudo-Label Selection
To prevent the propagation of errors between models, two specific selection strategies were developed:
- Hard-Soft Confidence Threshold (3D Network): The 3D network estimates its own reliability based on its performance on the sparse ground truth. If reliability is high, a "soft" (lower) threshold is used to retain more pseudo-labels; if low, a "hard" (higher) threshold ensures only the most certain predictions are passed to the 2D networks.
- Consistent Label Fusion (2D Networks): Because the two 2D networks observe the volume from different planes, the framework only accepts pseudo-labels where both 2D models agree, effectively filtering out view-specific noise.
Results and Discussion
The framework was evaluated on the MMWHS (Multi-Modality Whole Heart Segmentation) dataset, using only 16 labeled slices per volume (approximately 16% of the data used in fully supervised baselines).
Performance Metrics
The proposed method achieved a Dice score of 82.67%, significantly outperforming state-of-the-art semi-supervised methods like Cross-Pseudo Supervision (CPS), which reached 77.19%. Remarkably, the method slightly surpassed the fully-supervised upper bound of 81.69%, suggesting that the cross-teaching mechanism may provide a regularizing effect that prevents overfitting to specific slices.
Conclusion
By combining the "forest-level" context of 3D networks with the "tree-level" resolution of 2D networks, this framework demonstrates that high-fidelity medical segmentation is possible with a fraction of the manual labeling effort. This approach significantly lowers the barrier for deploying high-performance AI tools in clinical environments where expert time is at a premium.
Summary updated