Scribble-based Domain Adaptation via Co-segmentation

Reuben Dorent, Samuel Joutard, Jonathan Shapey, Sotirios Bisdas, Neil Kitchen, Robert Bradford, Shakeel Saeed, Marc Modat, Sébastien Ourselin, Tom Vercauteren

Abstract

Although deep convolutional networks have reached state-of-the-art performance in many medical image segmentation tasks, they have typically demonstrated poor generalisation capability. To be able to generalise from one domain (e.g. one imaging modality) to another, domain adaptation has to be performed. While supervised methods may lead to good performance, they require to fully annotate additional data which may not be an option in practice. In contrast, unsupervised methods don't need additional annotations but are usually unstable and hard to train. In this work, we propose a novel weakly-supervised method. Instead of requiring detailed but time-consuming annotations, scribbles on the target domain are used to perform domain adaptation. This paper introduces a new formulation of domain adaptation based on structured learning and co-segmentation. Our method is easy to train, thanks to the introduction of a regularised loss. The framework is validated on Vestibular Schwannoma segmentation (T1 to T2 scans). Our proposed method outperforms unsupervised approaches and achieves comparable performance to a fully-supervised approach.

Introduction

Deep Neural Networks (DNNs) are the current gold standard for medical image segmentation, yet they remain notoriously brittle when encountering a "domain shift." For instance, a model trained to segment tumors on contrast-enhanced T1-weighted (T1-c) MRI scans often fails significantly when applied to T2-weighted scans. This lack of generalization is a major barrier to clinical deployment.

Domain Adaptation (DA) techniques aim to mitigate this. Traditional Supervised DA requires clinicians to provide full boundary annotations for the new target domain, which is prohibitively expensive. Conversely, Unsupervised DA (UDA) attempts to align domains without any target labels, often through adversarial training. However, UDA is frequently unstable and can fail entirely when the visual discrepancy between modalities is large. This research proposes a practical middle ground: Weakly-supervised Domain Adaptation using simple "scribble" annotations.

Comparison of source domain (full masks) and target domain (scribbles) for Vestibular Schwannoma segmentation.

Methodology: Co-segmentation via Structured Learning

The core of the proposed method is framing domain adaptation as a dataset-level co-segmentation problem. This is achieved using a Conditional Random Field (CRF) that establishes relationships across the entire dataset.

The Graphical Model

The authors construct a graph where each node represents a voxel in the 3D images. Edges are defined in two ways:

  1. Intra-image edges: Connect voxels within the same scan to ensure local spatial and visual consistency.
  2. Inter-image edges: Connect voxels between different images (source and target). These edges are the conduit for knowledge transfer, allowing the fully-labeled source domain to guide the segmentation of the scribble-labeled target domain.
The proposed CRF framework connecting source and target domains across annotation and prediction layers.

Bridging the Modality Gap

A significant challenge in cross-modality DA is that identical tissues have different intensities in T1 vs. T2 scans. To overcome this, the authors compute cross-image similarity using deep features extracted from the network's penultimate layer. By operating in the feature space rather than the raw intensity space, the model can identify similar anatomical structures regardless of the imaging modality.

Single-Step Regularized Loss

Standard CRF integration often requires an alternating optimization process (e.g., Expectation-Maximization), which is slow and unstable. The authors introduce a mathematical relaxation that approximates the CRF optimization as a direct, single-step regularized loss function. This allows the entire system to be trained end-to-end using standard gradient descent, significantly improving stability and training speed.

Experimental Setup

The framework was evaluated on the segmentation of Vestibular Schwannoma (VS), a benign tumor of the inner ear.

  • Source Domain: 150 T1-c scans with full manual segmentations.
  • Target Domain (Training): 30 T2 scans annotated only with sparse scribbles. These scribbles covered only about 1% of the total scan volume and 7% of the tumor volume.
  • Target Domain (Testing): 50 T2 scans with full manual segmentations used for validation.

Results and Discussion

The proposed method demonstrated remarkable performance, effectively bridging the domain gap between T1 and T2 modalities. Quantitative evaluation shows that the proposed method not only outperforms unsupervised approaches—which largely failed due to the severe domain shift—but also slightly exceeds the performance of a model trained on fully annotated target data (83.4% vs 81.6% Dice score). This "better-than-supervised" result suggests that the rich information in the large, fully-labeled source dataset provides a strong regularizing effect that compensates for the sparsity of the target scribbles.

Conclusion

This work presents a robust and efficient framework for weakly-supervised domain adaptation in medical imaging. By leveraging sparse scribbles and a co-segmentation formulation based on deep feature similarity, the method achieves state-of-the-art results while drastically reducing the human annotation burden. The introduction of a regularized loss function makes the approach easy to implement and train, offering a practical path forward for deploying segmentation models across diverse clinical imaging protocols.

Summary updated

Abstract

Although deep convolutional networks have reached state-of-the-art performance in many medical image segmentation tasks, they have typically demonstrated poor generalisation capability. To be able to generalise from one domain (e.g. one imaging modality) to another, domain adaptation has to be performed. While supervised methods may lead to good performance, they require to fully annotate additional data which may not be an option in practice. In contrast, unsupervised methods don't need additional annotations but are usually unstable and hard to train. In this work, we propose a novel weakly-supervised method. Instead of requiring detailed but time-consuming annotations, scribbles on the target domain are used to perform domain adaptation. This paper introduces a new formulation of domain adaptation based on structured learning and co-segmentation. Our method is easy to train, thanks to the introduction of a regularised loss. The framework is validated on Vestibular Schwannoma segmentation (T1 to T2 scans). Our proposed method outperforms unsupervised approaches and achieves comparable performance to a fully-supervised approach.
BibTeX
@inproceedings{dorent2020scribble,
  author = {Reuben Dorent and Samuel Joutard and Jonathan Shapey and Sotirios Bisdas and Neil Kitchen and Robert Bradford and Shakeel Saeed and Marc Modat and Sébastien Ourselin and Tom Vercauteren},
  title = {{Scribble-based Domain Adaptation via Co-segmentation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  url = {https://openreview.net/forum?id=oRoZmJqEvs3},
}
Figure