Incorporating Cloud Distribution in Sky Representation

Kuan-Chuan Peng, Tsuhan ChenICCV 2013 · DOI

Abstract

Traditional analytical sky models simplify atmospheric conditions into a single global parameter, assuming uniform cloud distribution. This paper introduces a novel per-pixel sky representation that models the sky as a random field of indices. By leveraging the Igawa sky model within an energy minimization framework, the proposed method captures complex, non-uniform cloud patterns. Experimental results on the AMOS dataset demonstrate that this approach significantly improves sky reconstruction accuracy and enhances the reliability of sky-based geo-location.

Introduction

Accurate sky representation is a cornerstone of computer vision tasks such as lighting estimation and image rendering. However, standard models like the Perez or CIE models typically utilize a single "sky index" to describe the entire celestial hemisphere. This assumption of uniformity fails to account for patchy or localized cloud cover, which is the reality for most outdoor scenes. Consequently, researchers often discard cloudy images or treat clouds as noise. This paper argues that cloud distribution is a rich source of information—a "fingerprint" of a specific time and location—and proposes a method to model it explicitly.

Methodology

The core of the proposed approach is an extension of the Igawa sky model. The Igawa model defines sky intensity as a function of solar position and a sky index (ranging from 0 for overcast to 1 for clear). Instead of a global index, the authors treat the sky index as a per-pixel variable.

Visual representation of the Igawa sky model across different sky indices, ranging from overcast (0) to clear (1).

The process of generating a sky index image involves several key stages:

  1. Solar and Camera Estimation: Solar orientation is computed using timestamps and geographic coordinates. Camera parameters (zenith, azimuth, focal length) are estimated by maximizing the correlation between the input image and simulated sky maps.
  2. Energy Minimization: The task is formulated as a labeling problem. An energy function is defined with a unary term (consistency with observed intensity) and a binary term (a contrast-sensitive Potts model to ensure spatial smoothness while preserving cloud edges).
  3. Optimization: The $\alpha$-expansion algorithm, a graph-cut technique, is employed to find a strong local minimum for the pixel labels.
The proposed computational pipeline for estimating camera parameters and generating the per-pixel sky index image.

Experimental Evaluation

The model was evaluated using the AMOS (Archive of Many Outdoor Scenes) dataset, specifically a curated set of 198 cameras. The performance was measured against a baseline "uniform sky" model.

Key Results

  • Reconstruction Accuracy: The per-pixel model significantly reduced the Average Normalized Reconstructed Error (ANRE) from 0.627 (uniform model) to 0.262.
  • Geo-location Performance: In a task to predict camera location within a 200km radius based solely on sky appearance, the proposed model achieved 4.55% accuracy, a nearly nine-fold improvement over the uniform model's 0.51%.
  • Numerical Stability: Per-pixel modeling proved more robust to small errors in camera orientation. Unlike global models, where a slight shift might flip the entire sky state from clear to overcast, the local model confines the impact of perturbations to pixel boundaries.

Discussion and Limitations

While the model offers significant improvements, it relies on several physical simplifications. It lacks volumetric modeling, meaning it does not account for 3D cloud effects like internal shadows or complex light scattering within the cloud mass. Additionally, the current implementation is primarily luminance-based. Future work aims to incorporate chromatic information and more sophisticated atmospheric scattering models to further enhance realism and accuracy.

Conclusion

By moving from a global to a per-pixel sky representation, this research demonstrates that clouds are a valuable signal rather than noise. This approach not only enables more realistic image synthesis but also unlocks the potential of the sky as a reliable cue for geographic localization.

Summary updated

BibTeX
@inproceedings{peng2013incorporating,
  author = {Kuan-Chuan Peng and Tsuhan Chen},
  title = {{Incorporating Cloud Distribution in Sky Representation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2013},
  doi = {10.1109/iccv.2013.267},
}
Figure