Single-Image-Based Deep Learning for Segmentation of Early Esophageal Cancer Lesions

Haipeng Li, Dingrui Liu, Yu Zeng, Shuaicheng Liu, Tao Gan, Nini Rao, Jinlin Yang, Bing Zeng

TL;DR

Researchers have developed a "You-Only-Have-One" (YOHO) deep learning framework that segments early esophageal cancer lesions by training a unique neural network for every single patient image. By intentionally "over-fitting" to a synthetic dataset generated from just one image, the system achieves a mean Dice score of 0.888—a massive leap over the ~0.75 score typical of current state-of-the-art models.

What Problem Does This Paper Address?

Early Esophageal Cancer (EEC) is a major global health challenge, but early diagnosis significantly improves survival rates. Doctors use endoscopy to find these lesions, but segmenting them—precisely outlining the cancerous tissue—is notoriously difficult for AI.

Unlike polyps, which tend to have regular, bulbous shapes, EEC lesions are flat, irregularly shaped, and vary wildly between patients (see Figure 1). Furthermore, medical data is scarce due to privacy laws, and high-quality annotations from specialists are expensive. Standard deep learning models often fail to generalize to new patients, producing jagged, inaccurate outlines that aren't reliable enough for clinical use.

![Fig.

  1. EEC lesions (top row) are significantly more complex and varied in shape than polyps (bottom row).](/figures/art_8de94e7339d9465a9a2158d343d918cb)

Methods

The authors propose the YOHO (You-Only-Have-One) framework. Instead of training one massive model on thousands of patients, they train a small, specialized model for each individual image. The pipeline works in three main steps:

  1. Doctor-in-the-Loop Annotation: A physician roughly sketches the lesion area and takes a few small circular samples of the diseased tissue.
  2. Geometry-Based Rendering: This is the "secret sauce." The system takes those small tissue samples, cuts them into circles and triangles, and pastes them back onto the original image at random, non-overlapping positions. This creates a synthetic training set of 1,600 images derived entirely from the patient's own data.
  3. Edge-Enhanced UNet (eUNet): The model uses a modified UNet architecture. It doesn't just predict the lesion mask; it also features a dedicated "Edge Detector" branch. A "Consistency Loss" function ensures that the predicted mask and the predicted edges align perfectly, leading to much smoother and more accurate boundaries.

![Fig.

  1. The YOHO pipeline: from a single doctor-annotated image to a custom-trained eUNet.](/figures/art_ccdfae33ab6c4acaa5e61f8063366690)

Datasets

The study utilized three primary datasets:

  • EEC-2022: A new dataset created by the authors containing 1,236 high-resolution endoscopy images of early esophageal cancer, annotated under expert supervision.
  • CVC-ClinicDB (CVC-612): A standard open-access dataset for polyp segmentation.
  • Kvasir-SEG: A larger, medically verified polyp dataset.

The inclusion of polyp datasets was intended to show that YOHO generalizes well to other types of endoscopic lesion segmentation.

Key Insights and Ideas

  • Embracing Over-fitting: In most machine learning, over-fitting is a bug. In YOHO, it’s a feature. Since the model is only ever intended to be used on the image it was trained on, "over-fitting" to that specific patient's tissue texture and lighting ensures maximum local accuracy.
  • Privacy by Design: Because the network never sees data from other patients, there is zero risk of leaking sensitive medical information between cases.
  • Geometric Approximation: The authors found that training a model on simple circles and triangles of lesion tissue allows it to segment complex, arbitrary shapes. This is because any complex curve can be approximated by a series of short arcs and lines.
  • Edge Matters: By forcing the network to care about edges specifically (see Figure 3), the researchers eliminated the "jagged" artifacts common in other segmentation models.

![Fig.

  1. The eUNet architecture includes a parallel edge detector to refine the final segmentation.](/figures/art_5d07d929fd57411cbc04561aeddd2951)

Key Results

The YOHO framework significantly outperformed existing state-of-the-art (SOTA) models across the board:

  • EEC Segmentation: YOHO achieved a mean Dice score of 0.888, compared to scores between 0.68 and 0.73 for models like UNet++ and Polyp-PVT.
  • Polyp Segmentation: On the CVC-612 dataset, YOHO reached a Dice score of 0.940, surpassing the previous SOTA.
  • Comparison with SAM: YOHO outperformed Meta’s "Segment Anything Model" (SAM) by a large margin on EEC lesions, proving that specialized, small-data models can beat "foundation" models in niche medical tasks (see Figure 6).
  • Clinical Validation: In blind tests, four experienced endoscopists often found YOHO's segmentations to be as good as, or even better than, the manual ground-truth annotations.

![Fig.

  1. YOHO (bottom) provides much cleaner and more accurate segmentations than Meta's SAM (top).](/figures/art_ff180ba6251743e5a992d681cb9d45e3)

Limitations and Future Work

The primary drawback of YOHO is computational time. Because a new network must be trained for every image, it takes about 4.5 minutes to process a single case on a high-end GPU (NVIDIA RTX 2080 Ti). While this is acceptable for post-procedure analysis, it is currently too slow for real-time use during an active endoscopy.

Future work will focus on speeding up the training process and applying the YOHO "one-image-one-network" philosophy to other medical imaging modalities like CT and MRI scans.

Summary updated

Abstract

Accurate segmentation of lesions is crucial for diagnosis and treatment of early esophageal cancer (EEC). However, neither traditional nor deep learning-based methods up to today can meet the clinical requirements, with the mean Dice score - the most important metric in medical image analysis - hardly exceeding 0.75. In this paper, we present a novel deep learning approach for segmenting EEC lesions. Our approach stands out for its uniqueness, as it relies solely on a single image coming from one patient, forming the so-called "You-Only-Have-One" (YOHO) framework. On one hand, this "one-image-one-network" learning ensures complete patient privacy as it does not use any images from other patients as the training data. On the other hand, it avoids nearly all generalization-related problems since each trained network is applied only to the input image itself. In particular, we can push the training to "over-fitting" as much as possible to increase the segmentation accuracy. Our technical details include an interaction with clinical physicians to utilize their expertise, a geometry-based rendering of a single lesion image to generate the training set (the \emph{biggest} novelty), and an edge-enhanced UNet. We have evaluated YOHO over an EEC data-set created by ourselves and achieved a mean Dice score of 0.888, which represents a significant advance toward clinical applications.
BibTeX
@article{li2024single,
  author = {Haipeng Li and Dingrui Liu and Yu Zeng and Shuaicheng Liu and Tao Gan and Nini Rao and Jinlin Yang and Bing Zeng},
  title = {{Single-Image-Based Deep Learning for Segmentation of Early Esophageal Cancer Lesions}},
  journal = {IEEE Trans. Image Process.},
  year = {2024},
  doi = {10.1109/tip.2024.3379902},
}
Figure