Introduction
As High Dynamic Range (HDR) and Wide Color Gamut (WCG) televisions become the industry standard, the demand for up-converting legacy Standard Dynamic Range (SDR) content has surged. This process, known as SDR-to-HDRTV up-conversion, is an "ill-posed" inverse problem. It requires "hallucinating" information lost during the original SDR recording, such as clipped highlights in bright clouds or crushed details in deep shadows.
The "Training Gap" in Current Models
Existing deep-learning methods often struggle in real-world applications, frequently producing results that appear dim and desaturated. This paper identifies a fundamental "training gap" as the cause. Many current datasets use simplistic degradation models (like standard YouTube SDR conversion) that result in synthetic SDR versions that are over-brightened and over-saturate compared to real-world footage. When a network is trained to reverse this process, it inadvertently learns to darken and desaturate the image to match the target, leading to a poor viewing experience.
Proposed Methodology
To bridge this gap, the authors introduce a three-pronged solution:
1. Realistic Degradation Models
Instead of simple tone-mapping, the researchers developed models that simulate real-world SDR limitations, including hard-clipping (where brightness is capped at 100 nits) and gamut mapping. This forces the network to learn the actual recovery of lost information rather than performing a simple style transfer.
2. Luminance Segmented Network (LSN)
The proposed architecture, LSN, addresses the varying restoration needs of different image regions:
- Global Mapping Trunk: A lightweight series of 1x1 convolutions handles the overall color and tone shifts across the entire image.
- Transformer-style UNet Branches: Two specialized branches target low-luminance (shadows) and high-luminance (highlights) regions. By using Transformer blocks, the model captures long-range dependencies, enabling it to "inpaint" lost highlight details and effectively denoise dark areas.
3. Tailored HDR Metrics
Standard metrics like PSNR and SSIM often fail to capture the perceived quality of HDR content. The authors utilize specialized metrics such as FHLP (Fraction of HighLight Pixels) and FWGP (Fraction of Wide-Gamut Pixels) to measure how effectively the model utilizes the expanded HDR/WCG volume.
The HDRTV4K Dataset
A core contribution of this work is the HDRTV4K dataset, consisting of 3,878 high-quality pairs of HD and UHD frames. These frames were manually curated from diverse 4K content to ensure high "intra-frame diversity," featuring rich colors and complex lighting scenarios that challenge up-conversion models more effectively than previous benchmarks like KAIST or HDRTV1K.
Experimental Results
The LSN model demonstrates significant improvements over state-of-the-art (SOTA) methods. In subjective "side-by-side" tests on calibrated Sony HDR displays, users consistently rated the LSN output as superior to the original SDR, a feat shared only by professional commercial software like DaVinci Resolve.
Quantitatively, LSN achieves much higher scores in HDR utilization. As shown in the comparison above, metrics like the Brightness Index and Saturation Level for LSN are much closer to the Ground Truth (GT) than those of models like FMNet or HDRTVNet, which tend to produce dimmer results. Despite its use of Transformers, the LSN remains efficient with only ~325k parameters, making it a practical choice for 4K resolution processing.
Conclusion and Future Work
While LSN sets a new standard for practical up-conversion, challenges remain in handling extreme sensor noise in very large dark areas. Future research will focus on integrating more robust denoising priors and ensuring temporal consistency for video-based up-conversion to prevent flickering in recovered regions.
Summary updated