Abstract
Ultra-High-Definition (UHD) image enhancement is computationally demanding, necessitating a "Downsampling-Enhancement-Upsampling" paradigm. However, traditional resampling methods are independent of the enhancement model, leading to the loss of critical high-frequency details. This paper presents Learning Model-Aware Resampling (LMAR), a framework that correlates the resizer with a pre-trained enhancer. By utilizing Implicit Neural Representations and steganographic techniques, LMAR embeds model-specific compensatory information into downscaled images. Experimental results on 4K benchmarks demonstrate significant quality improvements, achieving up to 1.2dB PSNR gain without retraining the enhancement backbone.
Introduction
Processing 4K and 8K images directly with deep learning models often exceeds memory and computational limits. The standard industry solution involves downsampling the image, applying an enhancement model at a lower resolution, and then upsampling the result. A significant drawback of this approach is that standard resampling operators (e.g., Bicubic) are "blind" to the downstream enhancement model. They discard information indiscriminately, depriving the enhancer of specific cues needed for high-fidelity reconstruction.
Learning Model-Aware Resampling (LMAR)
To bridge the gap between resampling and enhancement, we propose LMAR. This method "empowers" existing interpolation operators by making them aware of the specific requirements of a pre-trained, frozen enhancement model.
Compensatory Kernel Estimation (CKE)
LMAR employs an Implicit Neural Representation (INR) to predict image-specific convolution kernels. Unlike fixed mathematical formulas, these kernels are learned to capture "compensatory information"—the specific details that the enhancement model requires to produce a high-quality UHD output.
Steganographic Resampling (SR)
The estimated compensatory information is embedded into the downscaled image using a steganographic approach. This creates a "model-favored" low-resolution input. To the human eye, the image appears normal, but it contains hidden features that the enhancement model can exploit for better restoration.
Experimental Evaluation
The effectiveness of LMAR was evaluated on UHD low-light enhancement datasets, including UHD-LOL4K and 4KIL.
Quantitative and Qualitative Results
LMAR consistently outperforms standard resampling across various scales (x1.5 to x5). On the UHD-LOL4K dataset, it achieved a PSNR improvement of up to 1.2dB at a x1.8 scale. The method is versatile, showing gains across different architectures such as CNNs and Transformers (e.g., Restormer). Qualitatively, LMAR produces sharper edges and superior global illumination compared to "blind" downsampling methods, which often result in blurriness or artifacts.
Efficiency and Compatibility
A key advantage of LMAR is its compatibility with frozen pre-trained models. It requires no retraining of the heavy enhancement backbone, only the lightweight LMAR module. The module adds minimal overhead, increasing model size by a small fraction (e.g., from 1.009MB to 1.314MB) with negligible impact on inference time.
Conclusion
LMAR addresses the fundamental disconnect in the "Downsampling-Enhancement-Upsampling" pipeline for UHD images. By making the resampling process model-aware, it preserves critical features that standard methods discard. This approach offers a practical and efficient way to boost the performance of existing UHD enhancement models without the need for expensive retraining.
Summary updated