Co-learning Semantic-Aware Unsupervised Segmentation for Pathological Image Registration

Yang Liu, Shi Gu

Medical image registration—the process of aligning two images into a single coordinate system—is a cornerstone of modern diagnostics and surgery. However, standard algorithms often struggle when one image contains a massive tumor and the other is a healthy "atlas" template. In these cases, algorithms frequently "hallucinate" distortions, attempting to force-fit pathological tissue into healthy structures where no spatial correspondence exists.

A recent study, "Co-learning Semantic-aware Unsupervised Segmentation for Pathological Image Registration," introduces GIRNet, a framework that addresses this by teaching three different neural networks to work together in a completely unsupervised "tri-net" loop. GIRNet simultaneously learns to register, segment, and inpaint pathological images, allowing for highly accurate registration without requiring manual labels or ground-truth masks.

The Challenge of Pathological Registration

The fundamental assumption of image registration is spatial correspondence: every point in Image A should have a corresponding point in Image B. Pathology, such as a brain tumor, breaks this assumption because it exists in the patient's scan but not in the healthy atlas.

Existing methods typically attempt to mask the tumor or synthesize a "normal" version of the pathological image. However, these approaches often require expensive manual labels or rely on complex loss functions that are difficult to tune. This creates a "chicken-and-egg" problem: accurate registration is needed to locate the lesion, but locating the lesion is necessary for accurate registration.

The GIRNet Framework

The authors propose GIRNet, based on the principles of Generation, Inpainting, and Registration. The framework consists of three modules trained in a "co-learning" manner:

  1. RegNet (The Register): Predicts the deformation field to align the images.
  2. SegNet (The Segmenter): Identifies the lesion area using Minimal Mutual Information (MMI). SegNet leverages semantic information from the registration process to distinguish between actual lesions and normal tissue variations.
  3. InpNet (The Inpainter): Fills the area identified by SegNet with healthy-looking tissue.
The GIRNet architecture featuring the co-learning loop between RegNet, SegNet, and InpNet modules.

In this workflow, SegNet identifies a lesion, InpNet replaces it with a "normal" version of that brain area, and RegNet then registers this "cleaned" image to the target. Because they are trained together, the registration helps the segmentation become more accurate, and better segmentation leads to cleaner inpainting, which in turn makes the registration more precise.

Experimental Results

The researchers evaluated GIRNet on several high-profile medical imaging datasets, including OASIS-1, BraTS2020, and a custom 3D Pseudo Brain MRI dataset where tumors were digitally "grafted" into healthy scans to provide a gold standard for registration error.

Registration Accuracy

GIRNet demonstrated state-of-the-art performance. On the pseudo-dataset, GIRNet achieved a Mean Deformation Error (MDE) of less than 1mm, significantly outperforming popular models like VoxelMorph and DIRAC. The inclusion of Histogram Matching (HM) proved vital for unsupervised success by minimizing domain differences between source and target intensities.

Mean Deformation Error (MDE) across different regions (IN, Near, Far) comparing GIRNet variants with baseline methods.

Unsupervised Segmentation

Despite never seeing a manual label during training, GIRNet achieved a Dice Similarity Coefficient (DSC) of 0.83 on the pseudo-dataset after post-processing. This high performance in a completely unsupervised setting highlights the power of the semantic-aware co-learning approach.

Conclusion and Future Directions

GIRNet represents a significant step forward in handling non-correspondence in medical imaging. By incorporating the registration path, the model learns that regions that cannot be mapped to the healthy template are likely lesions.

While highly effective, the authors note that registration accuracy remains dependent on inpainting quality. Future work will focus on enhancing the InpNet architecture to better handle complex domain differences and imaging modalities, further bridging the gap for clinical use in longitudinal tasks like tracking post-operative changes.

Paper Citation: Yang Liu and Shi Gu. "Co-learning Semantic-aware Unsupervised Segmentation for Pathological Image Registration." arXiv:2310.11040v3 [eess.IV], 18 Mar 2025.

Summary updated

Abstract

The registration of pathological images plays an important role in medical applications. Despite its significance, most researchers in this field primarily focus on the registration of normal tissue into normal tissue. The negative impact of focal tissue, such as the loss of spatial correspondence information and the abnormal distortion of tissue, are rarely considered. In this paper, we propose a novel unsupervised approach for pathological image registration by incorporating segmentation and inpainting. The registration, segmentation, and inpainting modules are trained simultaneously in a co-learning manner so that the segmentation of the focal area and the registration of inpainted pairs can improve collaboratively. Overall, the registration of pathological images is achieved in a completely unsupervised learning framework. Experimental results on multiple datasets, including Magnetic Resonance Imaging (MRI) of T1 sequences, demonstrate the efficacy of our proposed method. Our results show that our method can accurately achieve the registration of pathological images and identify lesions even in challenging imaging modalities. Our unsupervised approach offers a promising solution for the efficient and cost-effective registration of pathological images. Our code is available at https://github.com/brain-intelligence-lab/GIRNet.
BibTeX
@inproceedings{liu2023co,
  author = {Yang Liu and Shi Gu},
  title = {{Co-learning Semantic-Aware Unsupervised Segmentation for Pathological Image Registration}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  doi = {10.1007/978-3-031-43999-5\_51},
}
Figure