Hierarchical Coarse to Fine Segmentation for Volumetric Medical Imaging

This direction is split into other directions: coarse-to-fine segmentation (two-stage localization and segmentation). Its last version stays readable and citable here.

Version 1 of 1161 papers198 conceptsfrozen 2026-10-04

In medical imaging, researchers often face a "needle in a haystack" problem. A standard 3D CT or MRI scan contains millions of voxels, yet the target of interest—such as a small tumor, a cerebral aneurysm, or the pancreas—might occupy less than 1% of that volume. Processing the entire high-resolution volume at once is computationally prohibitive and statistically difficult due to extreme class imbalance.

To solve this, a dominant research direction has emerged: the hierarchical coarse-to-fine framework. Instead of asking a single model to find and delineate a structure simultaneously, this approach decomposes the task into two distinct stages: global localization and local refinement.

The Mechanism of Cascaded Segmentation

The core intuition behind this direction is that "where" an organ is and "exactly what its borders look like" are two different sub-problems that require different perspectives.

  1. The Coarse Stage (Localization): A first-stage network—often a downsampled 3D Fully Convolutional Network (FCN) or a 2D network—scans the entire volume. Its goal is not perfect precision, but rather to identify a "Region of Interest" (ROI). By working at a lower resolution, as seen in the work of Holger R Roth in Hierarchical 3D fully convolutional networks for multi-organ segmentation, the model gains a wide field of view, allowing it to use global anatomical context (like the position of the spine or ribs) to find the target.
  2. The Fine Stage (Refinement): Once the ROI is identified, the system crops the original high-resolution volume around that specific area. A second, more specialized network then operates exclusively on this "zoomed-in" patch. Because the background has been stripped away, the model can focus its entire learning capacity on the subtle gradients of the organ's boundaries.

Why the Hierarchy Works

This strategy succeeds because it addresses the three primary bottlenecks of volumetric AI:

1. Mitigating Class Imbalance In a full CT scan, "background" voxels vastly outnumber "organ" voxels. This often leads models to be overly conservative, failing to detect small structures to avoid false positives. By cropping the volume, the second stage sees a much more balanced distribution of classes. Alessandro Crimi and colleagues demonstrated in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries that this cascade is essential for segmenting brain tumors that occupy only a tiny fraction of the total brain volume.

2. Computational Efficiency Processing a $512 \times 512 \times 512$ volume at full resolution exceeds the memory limits of most GPUs. The coarse-to-fine approach, such as the ResDSN C2F framework, bypasses this by only applying high-resolution filters where they are needed. This allows for "dense" volumetric segmentation without the "hangover" of massive memory requirements, a point emphasized by Corentin Dancette in the development of RAPS-3D.

3. Filtering Anatomical Noise Complex environments, like the carotid artery or the pelvic bone, contain many "distractor" tissues that look similar to the target. A two-stage approach, like the GAPNet introduced by Zhang Lin in GAPNet: Granularity Attention Network with Anatomy-Prior-Constraint for Carotid Artery Segmentation, uses the first stage as an anatomical filter. By the time the data reaches the fine-grained network, the "leaky" non-target structures have already been pruned.

From Organs to "Pan-Cancer" Systems

While early work focused on single, large organs like the liver or spleen, the field has moved toward more complex "multi-organ" and "pan-cancer" challenges.

In the SegRap2023 benchmark, Xiangde Luo showed that while large organs are nearly "solved," small structures like the optic nerve or heterogeneous tumors still require the specialized attention of a coarse-to-fine pipeline. Similarly, the FLARE 2023 Challenge by Jun Ma highlighted that the two-stage strategy is the current standard for segmenting 13 abdominal organs and diverse lesions simultaneously, balancing the need for global anatomical consistency with local precision.

By mimicking the way a radiologist first glances at a scan to find an organ and then leans in to inspect its borders, hierarchical coarse-to-fine frameworks have become the backbone of modern clinical AI.

Papers in this version

Showing 60 of 161 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_hierarchical_coarse_to_fine_segmentation_3fb38c5f,
  title = {Hierarchical Coarse to Fine Segmentation for Volumetric Medical Imaging},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-coarse-to-fine-segmentation-for-volumetric-medical-imaging/rdr_3fb38c5f57ef41dea87ec8f3cc9b29f5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_hierarchical_coarse_to_fine_segmentation_3fb38c5f_v1,
  title = {Hierarchical Coarse to Fine Segmentation for Volumetric Medical Imaging},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/hierarchical-coarse-to-fine-segmentation-for-volumetric-medical-imaging/rdr_3fb38c5f57ef41dea87ec8f3cc9b29f5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}