KUNet: Imaging Knowledge-Inspired Single HDR Image Reconstruction

Hu Wang, Mao Ye, Xiatian Zhu, Shuai Li, Ce Zhu, Xue Li

Abstract

Single-image High Dynamic Range (HDR) reconstruction is a challenging ill-posed problem due to significant information loss in overexposed and underexposed regions of standard Low Dynamic Range (LDR) inputs. This paper introduces KUNet, a novel deep-learning architecture that integrates physical imaging principles directly into a UNet-style framework. By mathematically modeling the camera imaging pipeline, KUNet effectively constrains the solution space, leading to superior detail recovery and noise reduction. Experimental results on the NTIRE 2021 HDR Challenge and HDRTV datasets demonstrate that KUNet achieves state-of-the-art performance in both image and video reconstruction.

Introduction

Capturing the full dynamic range of real-world scenes remains a significant hurdle for standard consumer cameras. The resulting LDR images often suffer from "blown-out" highlights or "crushed" shadows. While deep learning has advanced the field of HDR reconstruction, many existing methods treat the process as a "black box" or attempt to reverse individual processing steps without physical constraints. This often results in artifacts, unrealistic color shifts, or excessive noise. KUNet addresses these issues by embedding the physics of image formation into the neural network's architecture, bridging the gap between data-driven learning and physical modeling.

Methodology: Knowledge-Inspired Architecture

The core of KUNet is the integration of a physical imaging formula into a specialized UNet structure. The authors derive a reconstruction formula based on sensor gain, exposure time, and noise, identifying three critical tasks: detail hallucination in overexposed areas, imaging parameter adjustment, and denoising.

The KUNet architecture featuring Knowledge-Inspired Blocks (KIB) and Knowledge-Inspired jump Connections (KIC).

Knowledge-Inspired Block (KIB)

Standard convolutional blocks are replaced by KIBs, which mirror the physical imaging formula $H_F = X \odot R + Y$. Each KIB contains three specialized sub-networks:

  • Recovery (R): Utilizes dense connections to reconstruct missing details in saturated regions.
  • Adjustment (X): Modulates features to simulate variations in sensor gain and exposure.
  • Denoising (Y): Specifically targets and suppresses imaging noise.

Knowledge-Inspired jump Connection (KIC)

Traditional skip connections in UNets can inadvertently propagate LDR noise to the output. KUNet introduces KICs, which employ a residual structure and cosine similarity filtering. This ensures that only relevant, noise-free features are mapped into the HDR space, facilitating a cleaner transition across the massive dynamic range gap.

Experiments and Results

KUNet was evaluated on two major benchmarks: the NTIRE 2021 HDR Challenge dataset and the HDRTV dataset.

Quantitative Performance

On the NTIRE 2021 dataset, KUNet achieved a Peak Signal-to-Noise Ratio (PSNR) of 44.83, surpassing previous state-of-the-art models like HDRUNet (44.10) and Deep SR-ITM (43.29).

Quantitative comparison of KUNet against state-of-the-art methods on the NTIRE 2021 HDR Challenge dataset.

On the HDRTV dataset, KUNet demonstrated exceptional color accuracy, achieving a PSNR of 37.78 and a significantly lower $\Delta I_{TP}$ compared to existing methods. Despite its sophisticated design, the model remains efficient with only 1.12 million parameters.

Discussion and Conclusion

By leveraging physical constraints, KUNet effectively limits the solution space for HDR reconstruction, preventing the generation of unrealistic pixel values. The "mask loss" used during training further ensures that the network prioritizes the most difficult overexposed regions. While KUNet excels in standard metrics, future work could incorporate perception-based loss functions to further improve visual quality as measured by HDR-VDP3. Overall, KUNet represents a significant step forward in combining domain knowledge with deep learning for high-fidelity image restoration.

Summary updated

BibTeX
@inproceedings{wang2022kunet,
  author = {Hu Wang and Mao Ye and Xiatian Zhu and Shuai Li and Ce Zhu and Xue Li},
  title = {{KUNet: Imaging Knowledge-Inspired Single HDR Image Reconstruction}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/196},
}
Figure