Fusion-Correction Network for Single-Exposure Correction and Multi-Exposure Fusion

Jin Liang, Anran Zhang, Jun Xu, Hui Li, Xiantong Zhen

Introduction

Capturing images with ideal exposure remains a significant challenge in digital photography. Even with advanced sensors, photographers often produce images that are either over-exposed ("blown out") or under-exposed ("crushed"). Traditionally, the computer vision community has addressed these issues through two distinct tasks: Single-Exposure Correction (SEC), which restores a single poorly exposed image, and Multi-Exposure Fusion (MEF), which merges multiple images taken at different exposure levels into a single high-dynamic-range (HDR) result.

In the paper "FCNet: A Convolutional Neural Network for Arbitrary-Length Exposure Estimation," researchers Jin Liang et al. propose a unified framework that bridges these two domains. Their model, FCNet, is designed to handle image sequences of any length—from a single snapshot to a multi-frame burst—to produce a perfectly exposed output.

The Unified Exposure Problem

Existing methods often treat SEC and MEF as isolated problems. Standard MEF algorithms typically assume that at least one part of the input sequence is well-exposed. However, if an entire sequence is under-exposed, traditional MEF fails because there is no "good" data to fuse. Conversely, SEC methods ignore the redundant information available in multi-image sequences. Furthermore, many deep learning models are rigid, requiring a fixed number of input frames.

FCNet is designed to process arbitrary-length sequences of varying exposure levels to produce a single, well-exposed result.

FCNet addresses these limitations by exploiting the correlations between fusion and correction, allowing for a flexible input pipeline that scales from one to many images.

Methodology: The Laplacian Pyramid Approach

The core of FCNet is built upon a Laplacian Pyramid (LP) decomposition. This strategy breaks an image into multiple scales: a low-resolution "base" layer representing global illumination and several "detail" layers representing high-frequency textures.

At each level of the pyramid, the network employs a "Fusion-then-Correction" pipeline:

  1. Fusion Block: When multiple images are present, this block predicts weight maps to merge them efficiently using a "Downsample-Execute-Upsample" strategy to minimize computational cost.
  2. Correction Block: A UNet-like structure adjusts the luminance and color of the fused result to match a well-exposed target.
  3. Recursive Composition: The corrected output from a coarser level is upsampled and combined with details from the next level until the full-resolution image is reconstructed.

To support arbitrary sequence lengths, the network concatenates inputs along the batch dimension and utilizes a normalization step within the fusion block to ensure weight maps always sum to unity, regardless of the number of input frames.

Results and Performance

The researchers evaluated FCNet using the MIT-Adobe FiveK dataset, consisting of over 24,000 images. The model demonstrated state-of-the-art performance across both SEC and MEF tasks.

An example of the Multi-Exposure Fusion (MEF) output generated by the FCNet framework, showing balanced exposure and preserved detail.

Quantitative and Qualitative Gains

  • Superior Accuracy: In MEF tasks, FCNet achieved a Peak Signal-to-Noise Ratio (PSNR) of 20.81, significantly outperforming existing models like MEF-Net (18.93) and MEF-GAN (17.99).
  • Computational Efficiency: By placing complex correction logic at lower pyramid levels, FCNet reduces Floating Point Operations (FLOPs). It can process a 1024x1024 image sequence in approximately 0.033 seconds, making it roughly three times faster than its closest competitors.
  • Visual Consistency: The introduction of a Pyramid Spatial Consistency Loss ($L_{ps}$) helps prevent common artifacts such as "color spotting" or unnatural whitening often seen in deep-learning-based exposure tools.

Conclusion

FCNet represents a significant step toward a universal exposure correction tool. By unifying fusion and correction within a multi-scale framework, it provides a versatile, efficient, and high-performance solution for photographers. While future work may address dynamic scenes with moving objects, FCNet currently sets a new benchmark for static image exposure enhancement.

Summary updated

Abstract

The photographs captured by digital cameras usually suffer from over or under exposure problems. For image exposure enhancement, the tasks of Single-Exposure Correction (SEC) and Multi-Exposure Fusion (MEF) are widely studied in the image processing community. However, current SEC or MEF methods are developed under different motivations and thus ignore the internal correlation between SEC and MEF, making it difficult to process arbitrary-length sequences with improper exposures. Besides, the MEF methods usually fail at estimating the exposure of a sequence containing only under-exposed or over-exposed images. To alleviate these problems, in this paper, we develop a novel Fusion-Correction Network (FCNet) to tackle an arbitrary-length (including one) image sequence with improper exposures. This is achieved by fusing and correcting an image sequence by Laplacian Pyramid (LP) image decomposition. In each LP level, the low-frequency base component of the input image sequence is fed into a Fusion block and a Correction block sequentially for consecutive exposure estimation, implemented by alternative exposure fusion and correction. The exposure-corrected image in current LP level is upsampled and fused with the high-frequency detail components of the input image sequence in the next LP level, to output the base component for the Fusion and Correction blocks in next LP level. Experiments on the benchmark dataset demonstrate that our FCNet is effective on arbitrary-length exposure estimation, including both SEC and MEF. The code is publicly released at https://github.com/NKUJinLiang/FCNet.
BibTeX
@misc{liang2022fusion,
  author = {Jin Liang and Anran Zhang and Jun Xu and Hui Li and Xiantong Zhen},
  title = {{Fusion-Correction Network for Single-Exposure Correction and Multi-Exposure Fusion}},
  year = {2022},
  eprint = {2203.03624},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.03624},
}
Figure