Abstract
Multi-exposure High Dynamic Range (HDR) reconstruction often struggles with dynamic scenes where motion blur and extreme exposures lead to information loss. Traditional deep learning models focus on pixel-level alignment but lack high-level scene understanding. This paper presents a novel Semantic Knowledge Transfer (SKT) framework that enhances existing HDR models by distilling semantic priors from a teacher model. By aligning latent representations and using semantic-guided losses, the framework significantly improves reconstruction quality across various architectures. Crucially, the semantic components are discarded after training, providing performance gains with zero additional inference cost.
Introduction
High Dynamic Range (HDR) imaging aims to capture the full range of luminance in a scene by combining multiple Standard Dynamic Range (SDR) images with different exposures. However, this process is inherently ill-posed, especially in dynamic environments. Moving objects cause "ghosting" artifacts, and extreme exposures can lead to saturated regions where texture is lost. While modern CNNs and Transformers have improved alignment and fusion, they often lack a global context. For instance, knowing that a saturated region belongs to a "car" or "sky" could provide critical priors for more accurate restoration.
Methodology
The proposed Semantic Knowledge Transfer (SKT) framework utilizes a teacher-student distillation approach to inject high-level semantic wisdom into standard HDR reconstruction models.
The framework consists of four primary components:
- Original Reconstruction Model (ORM): The "student" model, which can be any existing HDR architecture.
- Semantic Priors Guided Reconstruction Model (SPGRM): The "teacher" model, which incorporates a Semantic Knowledge Bank (leveraging FastSAM) to extract features and segmentation maps.
- Semantic Knowledge Alignment Module (SKAM): This module bridges the domain gap between the student (often operating in RAW/Linear space) and the teacher (using sRGB-based semantics). It employs a masked feature mixer to align internal representations in a shared latent space.
- Semantic-Guided Histogram Loss: This loss function ensures color consistency across specific object instances, helping the model handle varying brightness levels more effectively.
A key advantage of this framework is that the semantic extraction network and the teacher model are only used during training. During inference, the "boosted" student model operates independently, maintaining its original computational efficiency.
Experimental Results
The framework was rigorously tested on standard benchmarks like Kalantari and challenging RAW-format datasets like BracketIRE. The results demonstrate consistent improvements across different baseline architectures, including AHDR, HDR-Trans, and TMRNet.
Quantitatively, the framework achieved a PSNR improvement of up to 2.46 dB for the AHDR baseline on the BracketIRE dataset. Perceptually, the LPIPS scores dropped significantly, indicating more natural-looking results. Qualitatively, as shown in the visual comparisons, the "boosted" models (denoted with a '+') excel at removing ghosting artifacts and recovering fine details in overexposed areas where baseline models typically fail.
Conclusion
The Semantic Knowledge Transfer framework offers a powerful, architecture-agnostic way to improve HDR reconstruction. By leveraging high-level semantics during training, models gain a deeper understanding of scene structure without any inference-time penalty. While the training process is more computationally intensive and currently faces some stability issues with GAN-based models, the significant gains in both quantitative metrics and visual quality make it a promising direction for future HDR research.
Summary updated