TL;DR
This paper introduces a novel tone mapping algorithm for High Dynamic Range (HDR) images, inspired by the "anchoring theory" of human lightness perception. By decomposing images into "frameworks" of consistent luminance and using a local "highest luminance" anchoring rule, the method aims to reproduce perceived lightness more accurately than existing techniques, preserving local contrast and natural colors.
What Problem Does This Paper Address?
Displaying High Dynamic Range (HDR) images on standard Low Dynamic Range (LDR) displays presents a significant challenge: how to compress the vast range of luminances in the real world into the limited range of a display while preserving the perceived lightness of objects. This is more than just a technical scaling problem; it's about mimicking how the human visual system (HVS) interprets lightness. The HVS exhibits "lightness constancy," meaning it perceives the lightness of an object relative to its surroundings and the overall illumination. However, this constancy can fail in certain conditions (e.g., simultaneous contrast, Gelb illusion), and these failures are important to consider for accurate image reproduction. Existing tone mapping methods often struggle to account for these perceptual nuances, leading to unnatural-looking images or a loss of detail. This paper aims to bridge this gap by grounding its tone mapping approach in a well-established theory of human lightness perception.
Methods
The core of the proposed method is a tone mapping algorithm that leverages the "anchoring theory" of lightness perception. The process involves several key steps:
- Image Decomposition into Frameworks: The HDR image is decomposed into overlapping "frameworks," which are regions of consistent illumination. This is achieved by analyzing the histogram of the image's luminance in log10 space. A K-means clustering algorithm is used to identify initial centroids, which are then refined to meet constraints (e.g., minimum difference between centroids, non-empty areas). Each pixel is assigned a probability of belonging to each framework.
- Framework Articulation: The "articulation" of a framework, representing how strongly pixels are anchored within it, is estimated based on its dynamic range. Frameworks with higher dynamic ranges have higher articulation, meaning their local anchoring is more influential. This articulation factor is used to weight the pixel probabilities.
- Anchor Estimation: For each framework, a local "anchor" value (perceived white) is estimated using the "highest luminance rule." Specifically, the algorithm computes the 95th percentile of luminance within the framework, effectively ignoring the brightest 5% of pixels to avoid issues with self-luminous highlights.
- Merging Frameworks: Finally, the global lightness of each pixel is computed by merging the individual framework lightness values, weighted by their respective probability maps and local anchors. This merging process is done linearly, preserving original colors.
Datasets
The paper uses several HDR images from the OpenEXR samples and their own database for demonstration and evaluation. Specific datasets for training are not mentioned, as the method is not a learning-based approach.
Key Insights and Ideas
- Anchoring Theory as a Foundation: The paper highlights that a robust tone mapping algorithm must go beyond simple luminance scaling and incorporate principles of human visual perception, specifically lightness constancy and its failures.
- Frameworks for Local Perception: Decomposing an image into "frameworks" of consistent illumination allows for localized application of perceptual rules, mirroring how the HVS processes complex scenes.
- Highest Luminance Rule for Anchoring: The authors argue for the superiority of the "highest luminance rule" (anchoring to perceived white) over the "average luminance rule" (anchoring to middle gray) for tone mapping, citing experimental evidence and demonstrating its effectiveness in simulating perceptual phenomena like the Gelb effect.
- Preservation of Local Contrast and Color: By using a linear merging process and anchoring to local luminance values, the method aims to preserve local contrast and natural colors, which are often compromised in other tone mapping techniques.
Key Results
- The proposed algorithm successfully reproduces the perceived lightness in simulations of the Gelb effect, outperforming existing tone mapping algorithms like photographic tone reproduction and fast bilateral filtering in this perceptual test.
- The framework decomposition method effectively identifies regions of common illumination in complex scenes, even without semantic information.
- The tone mapping results on various HDR images demonstrate good preservation of local contrast and natural colors.
- The algorithm is computationally efficient, with the majority of the time spent on framework decomposition.
Limitations and Future Work
- Limited to Static Images: The current approach is designed for static HDR images. The authors propose extending the technique to handle dynamic sequences by smoothing local anchor values over time, while also addressing potential ghosting artifacts.
- Assumption of Diffuse Surfaces: The Gilchrist's model, which the anchoring theory is based on, generally assumes diffuse surfaces. While the authors apply it to complex scenes with self-luminous areas, they acknowledge that a perceptual model for natural scenes with large self-luminous surfaces is still lacking.
- Subjectivity of Direct Comparison: The authors note that direct picture-to-picture comparisons with previous tone mapping methods can be subjective.
Summary updated