Abstract: Transferring a low-dynamic-range (LDR) image to a high-dynamic-range (HDR) image, which is the so-called inverse tone mapping (iTM), is an important imaging technique to improve visual effects of imaging devices. In this paper, we propose a novel deep learning-based iTM method, which learns an inverse tone mapping network with a generative adversarial regularizer. In the framework of alternating optimization, we learn a U-Net-based HDR image generator to transfer input LDR images to HDR ones, and a simple CNN-based discriminator to classify the real HDR images and the generated ones. Specifically, when learning the generator we consider the content-related loss and the generative adversarial regularizer jointly to improve the stability and the robustness of the generated HDR images. Using the learned generator as the proposed inverse tone mapping network, we achieve superior iTM results to the state-of-the-art methods consistently.
Introduction
High-dynamic-range (HDR) content is essential for modern ultra-high-definition displays, yet the vast majority of existing media is still in low-dynamic-range (LDR). Manually converting this legacy content is impossible at scale, necessitating automated Inverse Tone Mapping (iTM).
The challenge lies in the complexity of the mapping. Traditional iTM methods often rely on simplified, empirical formulas for luminance and color that fail to capture the non-linear nature of light or the complex correlations between color channels. Recent deep learning attempts have improved results but often struggle with "over-exposure" artifacts, color saturation issues, or a lack of fine detail (over-smoothing).
Proposed Method: The iTMN Framework
The authors propose a framework called iTMN (Inverse Tone Mapping Network) based on the Generative Adversarial Network (GAN) architecture. The system consists of two primary components:
- The Generator ($G$): A U-Net-based architecture acts as the "editor." It uses an encoder-decoder structure with skip connections to preserve multi-scale information. It consists of 10 layers (5 convolution, 5 deconvolution) using LeakyReLU activations and Batch Normalization.
- The Discriminator ($D$): A simple CNN that acts as a "critic," learning to distinguish between ground-truth HDR images and the "fake" ones produced by the generator.
Hybrid Loss Function
The "secret sauce" of iTMN is its hybrid loss function, which moves beyond standard Mean Squared Error (MSE) to ensure visual realism:
- MSE: Ensures pixel-level accuracy.
- dMSE (Differential MSE): Calculates the difference in horizontal and vertical gradients between the generated and real images. This forces the network to maintain sharp edges and fine textures, suppressing over-smoothness.
- Adversarial Regularizer: This term pushes the generator to produce images that match the statistical distribution of real HDR data, ensuring the output "looks" authentic rather than just minimizing a mathematical distance.
Experimental Setup
Because no standard large-scale dataset for iTM existed, the authors built their own training set of 2,660 HDR images. These were converted to the RGB channel, resized to 512x512, and quantified in 10-bit following the BT.2020 color gamut standard. To create input pairs, various standard tone mapping operators (like ReinhardTMO) were applied to the HDR images to simulate LDR counterparts.
Results and Discussion
The iTMN was compared against several baselines, including Huo, Kovaleski (KO), and Endo (DrTM).
Quantitative Performance
iTMN achieved the highest scores across three major metrics:
- HDR-VDP-2: 73.77 (A metric specifically designed to predict human visual quality for HDR).
- mPSNR: 21.61 (Pixel-wise quality).
- SSIM: 0.8098 (Structural similarity).
Qualitative Analysis and Ablation Studies
Visually, the iTMN results showed better contrast and fewer color distortions compared to traditional operators, which often suffered from serious over-saturation or "washed out" appearances. Ablation studies proved the necessity of each component. Removing the dMSE loss (NoDMSE) or the adversarial regularizer (NoAdvReg) led to a significant drop in SSIM and HDR-VDP-2 scores, confirming that both fine-detail preservation and distribution matching are vital.
Conclusion
This paper demonstrates that treating iTM as a supervised image-to-image translation task with an adversarial regularizer yields superior results. By penalizing errors in image gradients (dMSE), the model respects the "flow" and "edges" of the scene, where human vision is most sensitive. While the current work focuses on static images, extending this to video HDR processing—handling temporal consistency to prevent flickering—remains an important area for future research.
Summary updated