Deep Low-Contrast Image Enhancement using Structure Tensor Representation

Hyungjoo Jung, Hyunsung Jang, Namkoo Ha, Kwanghoon SohnAAAI 2021 · OpenReview

Abstract

We present a new deep learning framework for low-contrast image enhancement, which trains the network using the multi-exposure sequences rather than explicit ground-truth images. The purpose of our method is to enhance a low-contrast image so as to contain abundant details in various exposure levels. To realize this, we propose to design the loss function using the structure tensor representation, which has been widely used as high-dimensional image contrast. Our loss function penalizes the difference of the structure tensor between the network output and the multi-exposure images in a multi-scale manner. Eventually, the network trained by the loss function produces a high-quality image approximating the overall contrast of the sequence. We provide in-depth analysis on our method and comparison with conventional loss functions. Quantitative and qualitative evaluations demonstrate that the proposed method outperforms the existing state-of-the-art approaches in various benchmarks.

Introduction

Enhancing low-contrast images is a classic computer vision problem with significant practical applications. However, defining "ideal" contrast is inherently subjective, making it difficult to establish a single ground truth for training deep learning models. Traditional methods like histogram equalization often result in artificial appearances, while Retinex-based models can amplify noise.

Modern deep learning approaches typically rely on paired datasets—degraded images matched with expert-retouched versions. This reliance is problematic as human experts often disagree, and training on a single target can introduce artifacts like color distortion or halos. This paper addresses this gap by proposing a framework that leverages the rich information found in multiple exposures of the same scene, supervising the network without forcing it to replicate a single, potentially flawed, target image.

Methodology

Structure Tensor Representation

Rather than relying on pixel-to-pixel comparisons like Mean Squared Error (MSE), the researchers utilize Structure Tensors. A structure tensor is a 2x2 matrix describing local gradient information, capturing both the direction and magnitude of contrast. The loss function is designed to ensure the structural contrast of the output matches the combined contrast found across a sequence of exposures.

Multi-Scale Loss Function

The conceptual framework for variational image fusion using multi-exposure sequences to generate a high-quality fused result. Source, p. 3

The conceptual framework for variational image fusion using multi-exposure sequences to generate a high-quality fused result.

The core innovation is a multi-scale loss function that penalizes the difference between the structure tensors of the network's output and the multi-exposure training sequence. By calculating this loss across several scales (using Gaussian blurring and downsampling), the network captures both fine details and global color consistency. This ensures the final image maintains sharp edges while exhibiting natural-looking global luminance.

Network Architecture

The framework employs a Context Aggregation Network (CAN). Unlike standard encoder-decoder models that lose detail through downsampling, CAN uses dilated convolutions. This architecture allows for a massive receptive field—capturing the entire context of the image—while maintaining the original resolution throughout the enhancement process.

Experimental Results

Datasets and Evaluation

The model was trained on a large-scale multi-exposure dataset containing seven exposure levels per scene. Evaluation was conducted across four major benchmarks: SICE, Ma et al. (under-exposure), LIME (low-light), and DPED (low-end smartphone images). Performance was measured using the Naturalness Image Quality Evaluator (NIQE) and the Contrast Distortion Quality Evaluator (CDQE).

Performance Analysis

An example of an enhanced image produced by the proposed framework, demonstrating high detail and natural contrast. Source, p. 3

An example of an enhanced image produced by the proposed framework, demonstrating high detail and natural contrast.

Quantitatively, the proposed method achieved the best NIQE and CDQE scores across nearly all benchmarks, outperforming state-of-the-art models such as LIME and Retinex-Net. Visually, the results demonstrate superior color preservation and shadow detail. The method effectively avoids common pitfalls like the "washed-out" look or heavy noise, and it is robust against the "ghosting" artifacts often found in traditional multi-exposure fusion.

Conclusion

This research demonstrates that weak supervision using multi-exposure sequences is a powerful alternative to traditional ground-truth training. By treating contrast as a high-dimensional structural property via tensors, the network learns to synthesize high-quality results that surpass any single image in the training sequence. Future work may extend this framework to other image processing tasks where a single objective ground truth is difficult to define.

Summary updated

Abstract

We present a new deep learning framework for low-contrast image enhancement, which trains the network using the multi-exposure sequences rather than explicit ground-truth images. The purpose of our method is to enhance a low-contrast image so as to contain abundant details in various exposure levels. To realize this, we propose to design the loss function using the structure tensor representation, which has been widely used as high-dimensional image contrast. Our loss function penalizes the difference of the structure tensor between the network output and the multi-exposure images in a multi-scale manner. Eventually, the network trained by the loss function produces a high-quality image approximating the overall contrast of the sequence. We provide in-depth analysis on our method and comparison with conventional loss functions. Quantitative and qualitative evaluations demonstrate that the proposed method outperforms the existing state-of-the-art approaches in various benchmarks.
BibTeX
@inproceedings{jung2021deep,
  author = {Hyungjoo Jung and Hyunsung Jang and Namkoo Ha and Kwanghoon Sohn},
  title = {{Deep Low-Contrast Image Enhancement using Structure Tensor Representation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=t3bo1Nz8HV},
}
Figure