In traditional image restoration, we assume we know exactly how an image was "broken." To sharpen a blurry photo, we usually assume a specific mathematical blur—a Point Spread Function (PSF). To increase the resolution of a satellite image, we assume a fixed downsampling method.
However, real-world sensors are messy. Every camera has a unique spectral response and a slightly different lens blur. In hyperspectral imaging—where we capture dozens or hundreds of color bands—these unknowns are compounded. If our mathematical model of the sensor is even slightly off, the resulting high-resolution image will contain artifacts.
A growing body of research suggests a more elegant solution: stop guessing the degradation and start learning it. By treating the sensor's degradation process as a differentiable component of the neural network, models can "self-calibrate" to specific hardware without ever seeing a ground-truth high-resolution image.
The Degradation as a Proxy Layer
The core intuition behind this shift is to treat the physical process of image capture as a learnable layer. In a standard supervised setup, a network learns a mapping from Low-Resolution (LR) to High-Resolution (HR). In the unsupervised frameworks proposed by researchers like Lei Zhang and Ke Zheng, the network does the opposite: it learns to reconstruct the LR input from its own HR estimate.
To do this, the architecture includes a "degradation branch." As shown in UAL, this branch models the unknown sensor blur and noise as a convolutional layer with learnable weights. Instead of using a fixed Gaussian kernel, the network optimizes these weights during training. If the network’s estimate of the high-resolution image is correct, and its estimate of the sensor's blur is also correct, then applying that blur to the estimate should perfectly recreate the original low-resolution observation.
Self-Calibration through Reconstruction
This approach enables "blind" fusion—restoring images without knowing the sensor specifications. In HyCoNet, Ke Zheng and colleagues implement the PSF and the Spectral Response Function (SRF) as differentiable layers.
The training process becomes a balancing act:
- The Generator tries to create a high-resolution hyperspectral image.
- The Learnable Degradation Modules try to "break" that image back down to match the observed low-resolution data.
- The Loss Function minimizes the difference between the "re-degraded" image and the actual input.
Because the entire pipeline is differentiable, the error signal flows backward to update both the generator and the degradation parameters. As Danfeng Hong demonstrates in CUCaNet, this allows the model to operate entirely unsupervised. It doesn't need a library of "clean" images; it only needs the "dirty" ones it is trying to fix.
Unfolding Physical Priors
While these models are data-driven, they aren't "black boxes." Many of these architectures use algorithm unfolding, a technique where the layers of a neural network are designed to mirror the iterations of a classical optimization algorithm.
In MHF-net, Qi Xie and Deyu Meng treat the spectral response and spatial degradation matrices as learnable parameters within an unfolded iterative solver. This combines the interpretability of physical models—knowing that light must follow certain paths—with the flexibility of deep learning. Similarly, LDP-Net replaces rigid formulas for "graying" (converting color to grayscale) and "reblurring" with learnable modules, allowing the system to adapt to atmospheric conditions or specific satellite optics.
Generalizing to Universal Restoration
The power of learnable degradation extends beyond hyperspectral fusion into general image cleaning. Recent work on diffusion models, such as BIR-D by Siwei Tu and Weidong Yang, uses an optimizable convolutional kernel that evolves during the image generation process.
In this context, the model doesn't just learn the degradation once; it refines its "guess" about the blur or noise at every step of the denoising process. By minimizing the distance between the degraded input and the current estimate, the system can identify and reverse complex, mixed degradations—like a photo that is simultaneously blurry, noisy, and low-light—without any prior instructions on what the damage looks like.
By moving the degradation model from a fixed assumption to a learnable parameter, these researchers have turned a source of error into a source of information. The sensor itself becomes part of the learning process, leading to reconstructions that are more faithful to the physical reality of the scene.