Boosting HDR Image Reconstruction via Semantic Knowledge Transfer

Qingsen Yan, Tao Hu, Genggeng Chen, Wei Dong, Yan-Ning ZhangarXiv 2025 · DOI · arXiv

Abstract

Multi-exposure High Dynamic Range (HDR) reconstruction often struggles with dynamic scenes where motion blur and extreme exposures lead to information loss. Traditional deep learning models focus on pixel-level alignment but lack high-level scene understanding. This paper presents a novel Semantic Knowledge Transfer (SKT) framework that enhances existing HDR models by distilling semantic priors from a teacher model. By aligning latent representations and using semantic-guided losses, the framework significantly improves reconstruction quality across various architectures. Crucially, the semantic components are discarded after training, providing performance gains with zero additional inference cost.

Introduction

High Dynamic Range (HDR) imaging aims to capture the full range of luminance in a scene by combining multiple Standard Dynamic Range (SDR) images with different exposures. However, this process is inherently ill-posed, especially in dynamic environments. Moving objects cause "ghosting" artifacts, and extreme exposures can lead to saturated regions where texture is lost. While modern CNNs and Transformers have improved alignment and fusion, they often lack a global context. For instance, knowing that a saturated region belongs to a "car" or "sky" could provide critical priors for more accurate restoration.

Methodology

The proposed Semantic Knowledge Transfer (SKT) framework utilizes a teacher-student distillation approach to inject high-level semantic wisdom into standard HDR reconstruction models.

The Semantic Knowledge Transfer (SKT) framework architecture, featuring the Student (ORM), Teacher (SPGRM), and the Semantic Knowledge Alignment Module (SKAM).

The framework consists of four primary components:

  1. Original Reconstruction Model (ORM): The "student" model, which can be any existing HDR architecture.
  2. Semantic Priors Guided Reconstruction Model (SPGRM): The "teacher" model, which incorporates a Semantic Knowledge Bank (leveraging FastSAM) to extract features and segmentation maps.
  3. Semantic Knowledge Alignment Module (SKAM): This module bridges the domain gap between the student (often operating in RAW/Linear space) and the teacher (using sRGB-based semantics). It employs a masked feature mixer to align internal representations in a shared latent space.
  4. Semantic-Guided Histogram Loss: This loss function ensures color consistency across specific object instances, helping the model handle varying brightness levels more effectively.

A key advantage of this framework is that the semantic extraction network and the teacher model are only used during training. During inference, the "boosted" student model operates independently, maintaining its original computational efficiency.

Experimental Results

The framework was rigorously tested on standard benchmarks like Kalantari and challenging RAW-format datasets like BracketIRE. The results demonstrate consistent improvements across different baseline architectures, including AHDR, HDR-Trans, and TMRNet.

Qualitative comparison of various HDR reconstruction methods. The proposed "plus" versions (e.g., AHDR+, TMRNet+) show significant reductions in ghosting and better texture recovery in saturated regions.

Quantitatively, the framework achieved a PSNR improvement of up to 2.46 dB for the AHDR baseline on the BracketIRE dataset. Perceptually, the LPIPS scores dropped significantly, indicating more natural-looking results. Qualitatively, as shown in the visual comparisons, the "boosted" models (denoted with a '+') excel at removing ghosting artifacts and recovering fine details in overexposed areas where baseline models typically fail.

Conclusion

The Semantic Knowledge Transfer framework offers a powerful, architecture-agnostic way to improve HDR reconstruction. By leveraging high-level semantics during training, models gain a deeper understanding of scene structure without any inference-time penalty. While the training process is more computationally intensive and currently faces some stability issues with GAN-based models, the significant gains in both quantitative metrics and visual quality make it a promising direction for future HDR research.

Summary updated

Abstract

Recovering High Dynamic Range (HDR) images from multiple Standard Dynamic Range (SDR) images become challenging when the SDR images exhibit noticeable degradation and missing content. Leveraging scene-specific semantic priors offers a promising solution for restoring heavily degraded regions. However, these priors are typically extracted from sRGB SDR images, the domain/format gap poses a significant challenge when applying it to HDR imaging. To address this issue, we propose a general framework that transfers semantic knowledge derived from SDR domain via self-distillation to boost existing HDR reconstruction. Specifically, the proposed framework first introduces the Semantic Priors Guided Reconstruction Model (SPGRM), which leverages SDR image semantic knowledge to address ill-posed problems in the initial HDR reconstruction results. Subsequently, we leverage a self-distillation mechanism that constrains the color and content information with semantic knowledge, aligning the external outputs between the baseline and SPGRM. Furthermore, to transfer the semantic knowledge of the internal features, we utilize a Semantic Knowledge Alignment Module (SKAM) to fill the missing semantic contents with the complementary masks. Extensive experiments demonstrate that our framework significantly boosts HDR imaging quality for existing methods without altering the network architecture.
BibTeX
@misc{yan2025boosting,
  author = {Qingsen Yan and Tao Hu and Genggeng Chen and Wei Dong and Yan-Ning Zhang},
  title = {{Boosting HDR Image Reconstruction via Semantic Knowledge Transfer}},
  year = {2025},
  eprint = {2503.15361},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.15361},
}
Figure