Learning from Limited Labeled Medical Images

68 concepts68 papers1 proposal

Deep learning has transformed computer vision, but its success traditionally hinges on massive, cleanly annotated datasets like ImageNet. In medical imaging, this dependency creates a severe bottleneck. Expert radiologists' time is scarce and expensive, rare diseases inherently offer few examples, and strict privacy regulations prevent the seamless aggregation of patient data.

When deep networks are trained on small medical datasets, they tend to overfit to spurious shortcuts—such as the specific noise signature of a hospital's scanner—rather than learning true pathological features. As noted by Yueyuan Ao et al. in Feature Aggregation and Refinement Network for 2D Anatomical Landmark Detection, models must generalize across highly diverse anatomical presentations and varying image qualities without the luxury of massive training sets. To overcome this data scarcity, researchers have developed paradigms that extract maximum utility from unlabeled data, synthesize high-fidelity artificial samples, and optimize networks to learn from only a handful of examples.

Self-Supervision and Structural Invariance

One way to bypass the need for expert annotations is to force networks to learn the intrinsic geometry and topology of human anatomy from unlabeled scans. By designing pretext tasks rooted in anatomical structure, models can acquire robust visual representations before ever seeing a clinical label.

A powerful intuition here is that human anatomy follows consistent spatial arrangements. Kayhan Batmanghelich in Context Matters: Graph-based Self-supervised Representation Learning for Medical Images. leverages this by representing medical scans as graphs of anatomical regions. By training the network to understand the relationships between different anatomical nodes in a self-supervised manner, the model develops a deep contextual awareness that significantly boosts performance on downstream tasks like disease severity staging.

Alternatively, contrastive learning approaches teach models to recognize the same underlying clinical condition across different views or perturbations while distinguishing it from healthy tissue. For instance, Xiaoben Jiang et al. introduce a token projection mechanism in Contrastive learning with token projection for Omicron pneumonia identification from few-shot chest CT images to enhance few-shot classification accuracy by aligning patch-level representations across limited CT scans.

Standard Training:   Image ───► [ Deep Network ] ───► Requires Expert Label
                                        ▲
Self-Supervised:     Image ───► [ Spatial Graph ] ───► Learns Intrinsic Anatomy

Medical images also suffer from domain shifts caused by different scanner manufacturers or imaging protocols. To prevent a model from failing when deployed at a new hospital, Yue Zhi Russ Chua et al. develop contrast-invariant feature learning in Contrast Invariant Feature Representations for Segmentation and Registration of Medical Images. By forcing the network to map images to a contrast-agnostic latent space, downstream segmentation and registration tasks remain highly accurate even when trained on extremely limited data from a single protocol.

Generative Synthesis and Cross-Modality Mapping

When real data is scarce, generative models can fill the gaps by translating images across modalities or generating entirely synthetic training pairs. This allows networks to learn from simulated clinical variations that are missing from the original dataset.

For example, obtaining high-resolution positron emission tomography (PET) scans is often difficult due to high costs and radiation concerns. Shengye Hu et al. address this in Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis by using a bidirectional Generative Adversarial Network (GAN) framework. The model learns the underlying structural correlation between magnetic resonance (MR) images and PET scans, allowing it to synthesize realistic, diagnostically useful PET scans directly from widely available MR data.

For 3D imaging modalities like CT or MRI, manual voxel-wise annotation is exceptionally tedious. To automate this pipeline, Kun Han et al. propose MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generation. Instead of generating images and masks separately, the framework generates both the 3D medical volume and its corresponding segmentation mask simultaneously, providing an endless stream of perfectly labeled synthetic data to train downstream segmentation models.

Generative models can also act as advanced data augmentors within semi-supervised frameworks. Wenchi Ke et al. demonstrate in Incorporating Improved Sinusoidal Threshold-based Semi-supervised Method and Diffusion Models for Osteoporosis Diagnosis that diffusion models can generate highly realistic synthetic unlabeled data. When paired with a sinusoidal threshold decay mechanism for pseudo-labeling, the framework safely incorporates unlabeled data into the training loop without letting noisy predictions degrade the model's accuracy.

Maximizing Sample Efficiency via Few-Shot Learning

Few-shot learning directly optimizes models to generalize from a handful of annotated examples by reshaping the embedding space or intelligently selecting which samples deserve expert annotation.

In standard classification networks, decision boundaries can become highly unstable when trained on few samples. To enforce strict geometric separation in low-data regimes, Sharib Ali et al. introduce an additive angular margin loss in Additive Angular Margin for Few Shot Learning to Classify Clinical Endoscopy Images. By forcing the network to maximize the angular distance between different pathology classes in the latent space, the model learns highly discriminative features that prevent misclassification when encountering rare clinical endoscopy patterns.

Standard Softmax:       [ Class A ]  |  [ Class B ]   (Narrow, unstable boundary)

Additive Angular Margin: [ Class A ] ◄── Margin ──► [ Class B ] (Wide, robust boundary)

Instead of passively accepting a random subset of scarce data, active sample selection can maximize the utility of every single annotation budget. Longwei Wang et al. propose an expert-guided few-shot learning framework in Expert-Guided Explainable Few-Shot Learning with Active Sample Selection for Medical Image Analysis. This approach integrates human expertise directly into the active selection loop, ensuring that the model requests annotations only for the most informative and representative samples, thereby minimizing the total labeling effort required.

Finally, reinforcement learning can be used to optimize data augmentation strategies dynamically. Claudia Iriondo in Adversarial Policy Gradient for Deep Learning Image Augmentation develops an adversarial policy gradient framework that automatically learns to mask irrelevant or noisy regions of medical images. Rather than relying on manual, rule-based augmentations, the policy learns to remove uninformative background noise, forcing the downstream classifier to focus entirely on the critical diagnostic regions within small and uncertain datasets.

Key Foundational Work

Holistic segmentation of the lung in cine MRI (2017) — contributed 1 concept elements to this direction.
Real-Time Segmentation of Non-rigid Surgical Tools Based on Deep Learning and Tracking (2016) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_data_scarcity_in_medical_imaging_limited_1cc64e47,
  title = {data scarcity in medical imaging (limited labeled medical data)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/data-scarcity-in-medical-imaging-limited-labeled-medical-data/rdr_1cc64e47cb9a49babd32129afabbaac6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_data_scarcity_in_medical_imaging_limited_1cc64e47_v1,
  title = {data scarcity in medical imaging (limited labeled medical data)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/data-scarcity-in-medical-imaging-limited-labeled-medical-data/rdr_1cc64e47cb9a49babd32129afabbaac6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake