One-Shot Medical Landmark Detection

Qingsong Yao, Quan Quan, Li Xiao, S. Kevin Zhou

Abstract

The success of deep learning methods relies on the availability of a large number of datasets with annotations; however, curating such datasets is burdensome, especially for medical images. To relieve such a burden for a landmark detection task, we explore the feasibility of using only a single annotated image and propose a novel framework named Cascade Comparing to Detect (CC2D) for one-shot landmark detection. CC2D consists of two stages: 1) Self-supervised learning (CC2D-SSL) and 2) Training with pseudo-labels (CC2D-TPL). CC2D-SSL captures the consistent anatomical information in a coarse-to-fine fashion by comparing the cascade feature representations and generates predictions on the training set. CC2D-TPL further improves the performance by training a new landmark detector with those predictions. The effectiveness of CC2D is evaluated on a widely-used public dataset of cephalometric landmark detection, which achieves a competitive detection accuracy of 81.01% within 4.0mm, comparable to the state-of-the-art fully-supervised methods using a lot more than one training image.

Introduction

Deep learning has revolutionized medical image analysis, but it faces a significant "data hunger" problem. Training reliable landmark detectors—essential for tasks like surgical planning—typically requires radiologists to manually annotate hundreds of images. This process is expensive, time-consuming, and requires specialized expertise. This research addresses the extreme scenario of one-shot learning: can a model learn to find landmarks across a whole dataset if it is only shown a single annotated "template" image?

Methodology: The CC2D Framework

The authors propose a two-stage framework named Cascade Comparing to Detect (CC2D) to bridge the gap between a single annotation and robust dataset-wide detection.

Overview of the CC2D framework, illustrating the two-stage process from one-shot template to final detector.

Stage I: Self-Supervised Learning (CC2D-SSL)

Instead of attempting direct detection, the model learns to match anatomical patches. This stage mimics clinical intuition: practitioners often find landmarks by first identifying a general region (coarse) and then zooming in on the specific point (fine).

Detailed training and inference mechanisms of the CC2D-SSL stage, showing the cascade of similarity matrices across multiple scales.

The mechanism uses a "cascade" of feature representations. Deep layers of the neural network identify the general area of interest, while shallower layers refine the coordinates to pixel-level precision. This stage generates "pseudo-labels" (best-guess annotations) for the unlabeled training set by comparing the template to query images across multiple scales.

Stage II: Training with Pseudo-Labels (CC2D-TPL)

The pseudo-labels from Stage I are inherently noisy. However, deep learning models tend to learn consistent, "clean" patterns before fitting to noise. By training a standard Multi-task U-Net from scratch using these pseudo-labels, the framework effectively "denoises" the initial guesses. This second stage acts as a regularizer, resulting in a more robust final landmark detector.

Experimental Results

The framework was evaluated on the ISBI 2015 Cephalometric Landmark Detection dataset (400 skull radiographs, 19 landmarks). Using only one annotated image for the template, CC2D achieved:

  • Accuracy: A Successful Detection Rate (SDR) of 81.01% within a 4.0mm radius.
  • Efficiency: Performance comparable to state-of-the-art fully supervised methods trained on 150 labeled images.
  • Robustness: CC2D significantly outperformed other top-tier models when they were restricted to very few labels (e.g., 5 or 10), proving its strength in low-data regimes.

Conclusion and Future Work

CC2D demonstrates that anatomical invariance can be leveraged to achieve high-accuracy detection with minimal human intervention. While highly effective, the authors note that performance can vary based on the appearance difference between the template and query images. Future work may incorporate spatial relationships between landmarks (e.g., the relative position of the nose to the chin) to further improve accuracy and ensure anatomically plausible predictions.

Summary updated

Abstract

The success of deep learning methods relies on the availability of a large number of datasets with annotations; however, curating such datasets is burdensome, especially for medical images. To relieve such a burden for a landmark detection task, we explore the feasibility of using only a single annotated image and propose a novel framework named Cascade Comparing to Detect (CC2D) for one-shot landmark detection. CC2D consists of two stages: 1) Self-supervised learning (CC2D-SSL) and 2) Training with pseudo-labels (CC2D-TPL). CC2D-SSL captures the consistent anatomical information in a coarse-to-fine fashion by comparing the cascade feature representations and generates predictions on the training set. CC2D-TPL further improves the performance by training a new landmark detector with those predictions. The effectiveness of CC2D is evaluated on a widely-used public dataset of cephalometric landmark detection, which achieves a competitive detection accuracy of 81.01\% within 4.0mm, comparable to the state-of-the-art fully-supervised methods using a lot more than one training image.
BibTeX
@inproceedings{yao2021one,
  author = {Qingsong Yao and Quan Quan and Li Xiao and S. Kevin Zhou},
  title = {{One-Shot Medical Landmark Detection}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  doi = {10.1007/978-3-030-87196-3\_17},
}
Figure