Ultra-High-Definition Image HDR Reconstruction via Collaborative Bilateral Learning

Zhuoran Zheng, Wenqi Ren, Xiaochun Cao, Tao Wang, Xiuyi Jia

Introduction

While High Dynamic Range (HDR) displays have become commonplace, the majority of visual content remains in Standard Dynamic Range (SDR) or Low Dynamic Range (LDR). Converting these legacy images to HDR—a process known as inverse tone mapping—requires algorithms to intelligently reconstruct missing information in overexposed highlights and underexposed shadows. However, existing deep learning solutions often hit a "UHD Wall," where the computational demands of processing 4K (Ultra-High-Definition) images lead to excessive memory usage or slow processing speeds. Furthermore, fast upsampling techniques frequently introduce visual artifacts such as halos or "pseudo-edges."

This paper presents a Collaborative Bilateral Learning framework designed to overcome these challenges, achieving state-of-the-art reconstruction quality at a blistering 160 FPS for 4K images.

Collaborative Bilateral Learning

The proposed architecture moves away from standard pixel-to-pixel processing, instead focusing on a dual-path approach that decouples texture and color features to prevent interference during reconstruction.

Overview of the proposed Collaborative Bilateral Learning framework featuring dual-path processing and Tucker reconstruction.

Dual-Path Network and LeakyAdaIN

The model utilizes two distinct paths: one processing a grayscale version of the input to capture high-frequency texture and edges, and another processing a blurred color version to extract low-frequency chromatic features. To fuse these paths, the authors introduce LeakyAdaIN, an enhanced version of Adaptive Instance Normalization. LeakyAdaIN provides additional degrees of freedom, allowing the network to more accurately align content and style distributions for superior color fidelity.

3D Tucker Reconstruction

To achieve high speed, the network predicts a compact "bilateral grid" of local affine transforms rather than generating high-resolution pixels directly. To ensure these transforms vary smoothly and avoid blocky artifacts, the authors apply 3D Tucker decomposition to the grid. This acts as a low-rank prior that regularizes the grid, effectively smoothing the output and preventing the noise amplification common in traditional bilateral upsampling.

Experimental Results

The framework was evaluated on the MIT-Adobe FiveK and M-exposure datasets, as well as real-world 4K UHD images.

Quantitative and Qualitative Performance

As shown in the performance analysis below, the proposed method occupies the ideal top-left corner of the PSNR vs. Speed trade-off. It achieves a PSNR of 23.29 dB on the FiveK dataset, significantly outperforming previous real-time benchmarks like HDRNet (19.83 dB).

Performance comparison of PSNR vs. Speed, highlighting the efficiency of the proposed method compared to existing state-of-the-art models.

In terms of speed, the model processes UHD images in approximately 6ms on an NVIDIA Titan RTX. This efficiency allows for real-time 160 FPS performance, making it suitable for live video applications. Qualitatively, the method excels at recovering sharp details in saturated regions (e.g., clouds) without the color bleeding or halo artifacts seen in competing models.

Conclusion

By combining dual-path feature extraction with 3D Tucker-regularized bilateral grids, this research provides a robust solution for real-time UHD HDR reconstruction. The collaborative bilateral learning approach not only sets a new standard for inverse tone mapping but also shows great promise as a universal backbone for other high-resolution image restoration tasks such as dehazing and deblurring.

Summary updated

Abstract

Existing single image high dynamic range (HDR) reconstruction attempt to expand the range of luminance. They are not effective to generate plausible textures and colors in the reconstructed results, especially for high-density pixels in ultra-high-definition (UHD) images. To address these problems, we propose a new HDR reconstruction network for UHD images by collaboratively learning color and texture details. First, we propose a dual-path network to extract content and chromatic features at a reduced resolution of the low dynamic range (LDR) input. These two types features are used to fit bilatera-space affine models for real-time HDR reconstruction. To extract the main data structure of the LDR input, we propose to use 3D Tucker decomposition and reconstruction to prevents false edges and noise amplification in the learned bilateral grid. As a result, the high-quality content and chromatic features can be reconstructed capitalized on guided bilateral upsampling. Finally, we fuse these two full-resolution feature maps into the HDR reconstructed results. Our proposed method can achieve real-time processing for UHD image (about 160 fps). Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art HDR reconstruction approaches on public benchmarks and real-world UHD images.
BibTeX
@inproceedings{zheng2021ultra,
  author = {Zhuoran Zheng and Wenqi Ren and Xiaochun Cao and Tao Wang and Xiuyi Jia},
  title = {{Ultra-High-Definition Image HDR Reconstruction via Collaborative Bilateral Learning}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00441},
}
Figure