Self-Supervised Medical Image Reconstruction without Ground Truth

This direction is merged into another direction: paired data dependency (ground-truth data scarcity), self-supervised data partitioning. Its last version stays readable and citable here.

Version 1 of 1419 papers433 conceptsfrozen 2026-10-04

In supervised deep learning, the "ground truth" is the gold standard—a clean, high-resolution image that tells the model exactly what the output should look like. But in clinical medicine, ground truth is often a phantom. You cannot ask a patient to hold their breath for an hour to get a "static" cardiac MRI, nor can you ethically subject a patient to a high-dose CT scan just to provide a label for a low-dose one.

To bypass this bottleneck, a growing body of research focuses on self-supervised and unsupervised reconstruction. These methods allow neural networks to learn how to denoise, super-resolve, and reconstruct medical images using only the corrupted or undersampled data itself.

The Network as a Prior

The most intuitive leap in this field is the realization that a neural network's architecture itself can act as a filter. This is the core of the Deep Image Prior (DIP). Instead of training on thousands of images, the model is optimized to reconstruct a single image. Because convolutional networks are "biased" toward learning natural structures (low frequencies) before they learn noise (high frequencies), the optimization can be stopped before the noise is ever captured.

Jaejun Yoo and colleagues extended this to the temporal domain in Time-Dependent Deep Image Prior for Dynamic MRI. By using a network to represent a moving organ, like a beating heart, they showed that the model naturally enforces temporal continuity. Similarly, Jaeho Lee demonstrated in Zero-shot Blind Image Denoising via Implicit Neural Representations that coordinate-based MLPs (Implicit Neural Representations) can denoise images without any training data by exploiting this same "spectral bias."

Training on the Fly

If we don't have a clean reference, can we use the noisy data to supervise itself? This is the philosophy behind self-calibration.

In MRI recovery with self-calibrated denoisers without fully-sampled data, Philip Schniter and Sizhuo Liu introduced ReSiDe. Instead of using an external dataset, ReSiDe trains a denoiser on internal patches of the very scan it is trying to reconstruct. By iteratively refining the image and the denoiser simultaneously, the model "pulls itself up by its bootstraps."

This concept of "test-time adaptation" is further refined by Yutian Zhao in AdaptNet. Rather than relying on a static pre-trained model that might fail on a new patient, AdaptNet uses lightweight "AdaptBlocks" to fine-tune the reconstruction specifically for the unique physics of the scan currently in the machine.

Learning from Unpaired Distributions

Sometimes we have plenty of "clean" images and plenty of "noisy" images, but they aren't of the same patient. This is the unpaired translation problem.

Tian Xia and Sotirios A. Tsaftaris addressed this in Adversarial Pseudo Healthy Synthesis Needs Pathology Factorization. They used CycleGAN-style architectures to learn the distribution of "healthy" vs "pathological" anatomy. By factorizing the image into a healthy base and a pathology map, they could "remove" a tumor from a scan to see what the underlying healthy tissue should look like, despite never having seen that specific patient's healthy brain.

A similar logic applies to hardware differences. Aravind R. Krishnan used unpaired translation to harmonize CT scans across different vendors, ensuring that a scan from a GE machine looks like one from a Siemens machine, even without paired calibration data (Inter-vendor harmonization of Computed Tomography (CT) reconstruction kernels using unpaired image translation).

Joint Estimation of Motion and Physics

In many cases, the "noise" in an image isn't random; it's caused by unknown physical parameters, like patient movement or sensor sensitivity.

Weijie Gan and Yu Sun proposed MoDIR (Motion-Compensated Training for Deep Image Reconstruction without Ground Truth), which jointly learns to reconstruct the image and estimate the non-rigid deformation caused by motion. By treating motion as a learnable parameter rather than an error to be ignored, the model can "un-distort" the measurements during the reconstruction process.

This "calibration-free" approach is also seen in MRI. Tingjun Liu developed C-MSM, a diffusion-based model that estimates coil sensitivity maps—the "fingerprint" of the MRI hardware—directly from the undersampled data while reconstructing the image (Measurement Score-Based MRI Reconstruction with Automatic Coil Sensitivity Estimation).

Why It Works: The Physics Constraint

The common thread across these methods is the Forward Model. Unlike general image denoising (like removing grain from a photo), medical imaging follows strict physical laws (the Radon transform for CT, the Fourier transform for MRI).

Even without ground truth, we know that the final reconstructed image must be consistent with the raw measurements taken by the machine. By enforcing this "data consistency" as a hard constraint, researchers like Mingrui Geng (Unsupervised/Semi-supervised Deep Learning for Low-dose CT Enhancement) ensure that even an unsupervised model cannot simply "hallucinate" anatomical details. The physics of the scanner acts as the ultimate supervisor.

Papers in this version

Showing 60 of 419 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_self_supervised_medical_image_reconstruc_ba889f76,
  title = {Self-Supervised Medical Image Reconstruction without Ground Truth},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-medical-image-reconstruction-without-ground-truth/rdr_ba889f768d374b63abca83aad1341a30},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_self_supervised_medical_image_reconstruc_ba889f76_v1,
  title = {Self-Supervised Medical Image Reconstruction without Ground Truth},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-medical-image-reconstruction-without-ground-truth/rdr_ba889f768d374b63abca83aad1341a30/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}