Exposure-Structure Blending Network for High Dynamic Range Imaging of Dynamic Scenes

Sang-Hoon Lee, Haesoo Chung, Nam Ik ChoIEEE Access 2020 · DOI · OpenReview

Abstract

This paper presents a deep end-to-end network for high dynamic range (HDR) imaging of dynamic scenes with background and foreground motions. Generating an HDR image from a sequence of multi-exposure images is a challenging process when the images have misalignments by being taken in a dynamic situation. Hence, recent methods first align the multi-exposure images to the reference by using patch matching, optical flow, homography transformation, or attention module before the merging. In this paper, we propose a deep network that synthesizes the aligned images as a result of blending the information from multi-exposure images, because explicitly aligning photos with different exposures is inherently a difficult problem. Specifically, the proposed network generates under/over-exposure images that are structurally aligned to the reference, by blending all the information from the dynamic multi-exposure images. Our primary idea is that blending two images in the deep-feature-domain is effective for synthesizing multi-exposure images that are structurally aligned to the reference, resulting in better-aligned images than the pixel-domain blending or geometric transformation methods. Specifically, our alignment network consists of a two-way encoder for extracting features from two images separately, several convolution layers for blending deep features, and a decoder for constructing the aligned images. The proposed network is shown to generate the aligned images with a wide range of exposure differences very well and thus can be effectively used for the HDR imaging of dynamic scenes. Moreover, by adding a simple merging network after the alignment network and training the overall system end-to-end, we obtain a performance gain compared to the recent state-of-the-art methods.
BibTeX
@article{lee2020exposure,
  author = {Sang-Hoon Lee and Haesoo Chung and Nam Ik Cho},
  title = {{Exposure-Structure Blending Network for High Dynamic Range Imaging of Dynamic Scenes}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.3005022},
}

Loading related content...

Figure