Solving Inverse Problems Without Ground Truth Training Data

1,458 concepts1,192 papers10 proposals

Deep learning has transformed image restoration and reconstruction. However, standard supervised approaches rely on a luxury that many scientific and clinical fields cannot afford: large repositories of clean, fully-sampled "ground truth" data.

In magnetic resonance imaging (MRI), capturing a fully-sampled reference scan might require a patient to remain perfectly still for an impossible duration, as noted by Burhaneddin Yaman et al. in their work on Self-Supervised Physics-Based Deep Learning MRI Reconstruction Without Fully-Sampled Data. In other domains, such as underwater sonar imaging or nanoscale electron microscopy, noise-free references simply do not exist due to fundamental physical constraints.

To bypass this bottleneck, a powerful paradigm has emerged: solving inverse problems by learning directly from corrupted, incomplete, or undersampled observations. Instead of teaching a network what a "perfect" image looks like, these methods exploit the mathematical structure of the imaging process itself.

The Core Intuition: Physics as the Supervisor

An inverse problem can be formalized as recovering an underlying signal $x$ from a degraded measurement $y$:

$$y = A(x) + \epsilon$$

Here, $A$ represents the known forward physical operator (such as a Fourier transform in MRI, a projection matrix in CT, or a blur kernel in photography), and $\epsilon$ represents noise.

Supervised learning tries to learn a direct mapping $f_\theta(y) \approx x$. When $x$ is unavailable, we cannot compute a standard pixel-wise loss like $\|f_\theta(y) - x\|^2$. However, we do know the forward operator $A$. If we pass our network's estimate back through the physical operator, it should match our observed measurement:

$$A(f_\theta(y)) \approx y$$

By itself, this data-consistency constraint is insufficient because inverse problems are ill-posed—infinitely many clean images could produce the same degraded measurement. To resolve this ambiguity without ground truth, researchers leverage three primary mechanisms: measurement splitting, structural network priors, and algorithm unrolling.

Mechanism 1: Measurement Splitting and Self-Consistency

How do you calculate a loss when you only have an undersampled measurement? The key insight is to divide the available measurement data into two disjoint parts: a training set $y_{\text{train}}$ and a validation/loss set $y_{\text{loss}}$.

Burhaneddin Yaman et al. demonstrated this in Zero-Shot Self-Supervised Learning for MRI Reconstruction. In accelerated MRI, measurements are taken in the frequency domain (k-space). By partitioning a single patient's undersampled k-space data, the network is trained to reconstruct the full image using only $y_{\text{train}}$, but its performance is evaluated by how well its output matches the held-out $y_{\text{loss}}$ when passed back through the forward operator.

Because the noise in $y_{\text{train}}$ is statistically independent of the noise in $y_{\text{loss}}$, the network cannot minimize the loss by memorizing noise. It can only succeed by learning the underlying, coherent signal structure that bridges both subsets. This allows for scan-specific optimization without any external database.

Mechanism 2: Untrained Priors and Implicit Representations

Another strategy eliminates the training dataset entirely by embedding the image prior directly into the network's architecture. This approach relies on the discovery that deep network architectures are naturally biased toward generating clean, structured images rather than random noise.

For instance, Hannah Lawrence applied this concept to low-photon nanoscale imaging in Phase Retrieval with Holography and Untrained Priors: Tackling the Challenges of Low-Photon Nanoscale Imaging. By optimizing an untrained "Deep Decoder" network to fit a single noisy holographic measurement, the network acts as a regularizer. Because structured features are easier for the network to parameterize than chaotic noise, optimization converges on a clean reconstruction before it can overfit the noise.

Similarly, Implicit Neural Representations (INRs) treat an image not as a discrete grid of pixels, but as a continuous function mapping spatial coordinates $(x, y)$ to intensities. The authors of A scan-specific unsupervised method for parallel MRI reconstruction via implicit neural representation and Tianjing Zhang et al. in Cross-Scale Self-Supervised Blind Image Deblurring via Implicit Neural Representation leverage this coordinate-based continuity. The inherent smoothness of coordinate networks prevents high-frequency artifacts, enabling self-supervised deblurring and reconstruction directly at test time.

Mechanism 3: Deep Algorithm Unrolling

Instead of treating neural networks as black boxes, classical optimization algorithms can be "unrolled" into network architectures. This explicitly bakes the physics of data acquisition into the model.

In traditional compressed sensing, an image is reconstructed by iteratively balancing a data fidelity term and a hand-crafted regularizer (like sparsity). As shown by Alan Q. Wang et al. in Neural Network-Based Reconstruction in Compressed Sensing MRI Without Fully-Sampled Training Data (HQS-Net) and Shanshan Wang et al. in LANTERN: Learn analysis transform network for dynamic magnetic resonance imaging, each iteration of these algorithms can be mapped to a specific layer of a neural network.

The network learns the regularizer parameters from the available data, while the explicit data-consistency steps ensure the output never deviates from the physical measurements. This hybrid approach drastically reduces the amount of data required to train the network, allowing robust reconstruction even when fully-sampled references are impossible to obtain.

Overcoming Distribution Shift at Test Time

A major vulnerability of traditional supervised models is distribution shift: a model trained on a specific dataset often fails when encountering a new scanner configuration or an unexpected patient pathology.

To address this, Yutian Zhao et al. introduced AdaptNet in Test-Time Model Adaptation for Image Reconstruction Using Self-supervised Adaptive Layers. By inserting lightweight self-supervised layers into a pre-trained model, the network can fine-tune its parameters for each individual scan during inference.

Furthermore, Jason Hu et al. demonstrated in Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems that focusing generative models on local patches rather than global structures prevents the network from hallucinating incorrect anatomical features when confronted with unfamiliar data distributions. By anchoring the learning process to local textures and physical consistency, these models remain robust where traditional supervised networks fail.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_paired_data_dependency_ground_truth_data_a37b7b62,
  title = {paired data dependency (ground-truth data scarcity)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/paired-data-dependency-ground-truth-data-scarcity/rdr_a37b7b6274304d6b83ce9632f648a083},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_paired_data_dependency_ground_truth_data_a37b7b62_v1,
  title = {paired data dependency (ground-truth data scarcity)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/paired-data-dependency-ground-truth-data-scarcity/rdr_a37b7b6274304d6b83ce9632f648a083/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake