Self-Supervised Restoration for Real-World Sensor Noise

20 concepts20 papers1 proposal

In computer vision, the standard recipe for success is supervised learning: collect a massive dataset of "clean" ground-truth images, synthetically degrade them to create "noisy" inputs, and train a model to reverse the process. However, when these models are deployed on real-world sensors—like smartphone cameras, LiDAR, or high-speed spiking cameras—they often fail.

The reason is the domain gap. Synthetic noise models, such as Gaussian blur or bicubic downsampling, are mathematically convenient but physically inaccurate. As Dewil et al. argue, the "lesser evil" in training is not the lack of labels, but the presence of unrealistic synthetic data. To bridge this gap, a growing body of research focuses on learning directly from real-world, unpaired, and noisy data.

The Failure of Synthetic Supervision

Supervised models are highly sensitive to the distribution of their training data. If a model is trained to denoise Gaussian noise, it will struggle with the complex, non-linear noise patterns of a real CMOS sensor. Yuan et al. observe that supervised super-resolution models often amplify artifacts when they encounter real-world noise they weren't trained on.

Similarly, in 3D vision, models trained on perfect CAD models fail to generalize to the messy, occluded point clouds produced by LiDAR. Chen et al. and colleagues demonstrate that training on real-world scans—even without ground-truth completions—yields better results because the model learns the actual distribution of sensor noise and missing data.

Learning from Unpaired Distributions

If we don't have pairs of "noisy" and "clean" data, how can a model learn to restore an image? The intuition is to treat restoration as a domain translation problem.

One approach uses Generative Adversarial Networks (GANs) to map noisy inputs to a manifold of clean shapes or images. For example, Chen et al. use a GAN to ensure that the output of a point cloud completion network "looks like" a clean, complete shape, even though the network never sees a specific ground-truth pair.

To ensure the model doesn't just invent new content, researchers often use cycle-consistency. Kim et al. and Liu et al. employ "cycle-in-cycle" architectures where a model learns to degrade a clean image and restore a noisy one simultaneously. By enforcing that the restored image, when degraded again, matches the original input, the model learns the underlying physics of the sensor without needing explicit labels.

Physical Priors as Supervision

Another powerful alternative to labels is the use of physical priors—mathematical properties that we know must be true of the real world. Instead of comparing a model's output to a label, we penalize it if the output violates these laws.

  • Dark Channel Prior: Golts et al. use the observation that clean outdoor images typically have very low intensity in at least one color channel. By using this as a loss function, they train a dehazing network entirely on real hazy images.
  • Local Rigidity: In 3D motion estimation, Li et al. leverage the fact that most objects move as a collection of rigid parts. This "local rigidity prior" allows the model to generate its own pseudo-labels for scene flow, outperforming supervised models trained on synthetic data.
  • Motion Consistency: For ultra-high-speed spiking cameras, Xiong et al. exploit the fact that motion should be consistent across color channels and time, allowing for reconstruction without high-speed ground truth.

Exploiting Statistical Redundancy

Finally, self-supervision can be achieved by exploiting the statistical properties of noise itself. Noise is often independent across different samples or viewpoints, while the underlying signal is shared.

Hao Chen et al. introduce a framework that learns to denoise by comparing multiple noisy versions of the same scene, disentangling the shared features from the unique noise. Similarly, Pan et al. use a randomized sub-sampling strategy, treating the difference between sub-samples as a perturbation that guides the network toward the clean signal.

By shifting the training objective from "match this synthetic label" to "obey these physical and statistical constraints," these methods allow models to adapt to the specific idiosyncrasies of real-world sensors—from the Martian surface (Wang et al.) to high-speed neuromorphic streams (Chen et al.).

Key Foundational Work

Unpaired Point Cloud Completion on Real Scans using Adversarial Training (2020) — contributed 1 concept elements to this direction.
Unsupervised Single Image Super-Resolution Network (USISResNet) for Real-World Data Using Generative Adversarial Network (2020) — contributed 1 concept elements to this direction.
Unsupervised Real-World Super Resolution with Cycle Generative Adversarial Network and Domain Discriminator (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_self_supervised_domain_adaptation_ec87b83c,
  title = {self-supervised domain adaptation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-domain-adaptation/rdr_ec87b83c1afd4894bc0f6ce703fc92e4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_self_supervised_domain_adaptation_ec87b83c_v1,
  title = {self-supervised domain adaptation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-supervised-domain-adaptation/rdr_ec87b83c1afd4894bc0f6ce703fc92e4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake