Quantifying and Predicting Image Scenicness

Scott Workman, Richard Souvenir, Nathan Jacobs

Abstract While natural beauty is often considered a subjective property of images, in this paper, we take an objective approach and provide methods for quantifying and predicting the scenicness of an image. Using a dataset containing hundreds of thousands of outdoor images captured throughout Great Britain with crowdsourced ratings of natural beauty, we propose an approach to predict scenicness which explicitly accounts for the variance of human ratings. We demonstrate that quantitative measures of scenicness can benefit semantic image understanding, content-aware image processing, and a novel application of cross-view mapping, where the sparsity of ground-level images can be addressed by incorporating unlabeled overhead images in the training and prediction steps. For each application, our methods for scenicness prediction result in quantitative and qualitative improvements over baseline approaches.

Introduction

Computer vision has achieved remarkable success in identifying objective objects and scenes, yet quantifying subjective qualities like "natural beauty" remains a significant challenge. Traditional approaches often simplify beauty into a single average score, failing to account for the inherent variance in human perception. Furthermore, geographic mapping of scenicness faces a "sparsity problem": while popular landmarks are well-documented, remote or rural areas often lack ground-level photography. This research addresses these gaps by developing a deep learning framework to predict "scenicness" distributions and leveraging overhead satellite imagery to create dense, high-resolution beauty maps.

Methodology: Predicting Human Consensus

To capture the nuance of human opinion, the researchers utilized the ScenicOrNot (SoN) dataset, which contains over 185,000 images from Great Britain, each rated on a scale of 1–10. Rather than predicting a simple mean, the authors modified a GoogleNet architecture to predict the full probability distribution of human ratings.

Figure 4: Examples of scenic (a) and non-scenic (b) images from the ScenicOrNot dataset, paired with their respective human rating distributions (1–10).

By employing a Multinomial log-likelihood loss function, the model learns to "agree to disagree" in a manner consistent with human crowdsourcing. This approach proved significantly more accurate than standard regression models, matching human rating distributions in 58.4% of test cases compared to just 14.8% for mean-based baselines.

Semantic Markers and the "Bird's Eye" View

The model's analysis revealed clear semantic correlations with scenicness. High scenic scores are strongly associated with natural features such as summits, ridges, and lochs. Conversely, low scores are linked to man-made structures like roads, houses, and industrial infrastructure.

Figure 2: A visual contrast between highly scenic natural landscapes (left) and non-scenic industrial infrastructure (right), illustrating the semantic features the model identifies.

A key technical innovation of this work is the Cross-View Hybrid (CVH) Mapping. To solve the sparsity of ground-level photos, the researchers trained a network to predict scenicness from overhead satellite imagery (CVSoN dataset). They found that overhead patterns—such as the density of greenery versus asphalt—are powerful proxies for ground-level beauty. By combining these overhead features with available ground-level data, the CVH approach achieved an Area Under the Curve (AUC) of 68.51% in classifying scenic locations, outperforming traditional interpolation methods.

Applications and Results

The utility of these scenicness metrics extends beyond simple mapping:

  • Content-Aware Processing: The researchers developed a scenicness-aware cropping tool using Bayesian optimization. This "AI photographer" can automatically identify the most beautiful composition within a frame, often by cropping out distracting man-made elements like power lines or cars.
  • Environmental Health: Dense scenicness maps can help urban planners and public health researchers understand the relationship between environmental aesthetics and community well-being.

Conclusion and Future Work

While this study provides a robust framework for quantifying scenicness, it is currently limited by geographic and temporal biases. Future research could expand these models to different biomes (e.g., deserts or rainforests) and account for seasonal changes or time-of-day effects, such as the "golden hour," which significantly influence human perception. Ultimately, these high-resolution beauty maps could be integrated into GPS systems for scenic routing or used to monitor the impact of development on natural landscapes.

Summary updated

Abstract

While natural beauty is often considered a subjective property of images, in this paper, we take an objective approach and provide methods for quantifying and predicting the scenicness of an image. Using a dataset containing hundreds of thousands of outdoor images captured throughout Great Britain with crowdsourced ratings of natural beauty, we propose an approach to predict scenicness which explicitly accounts for the variance of human ratings. We demonstrate that quantitative measures of scenicness can benefit semantic image understanding, content-aware image processing, and a novel application of cross-view mapping, where the sparsity of ground-level images can be addressed by incorporating unlabeled overhead images in the training and prediction steps. For each application, our methods for scenicness prediction result in quantitative and qualitative improvements over baseline approaches.
BibTeX
@misc{workman2016quantifying,
  author = {Scott Workman and Richard Souvenir and Nathan Jacobs},
  title = {{Quantifying and Predicting Image Scenicness}},
  year = {2016},
  eprint = {1612.03142},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1612.03142},
}
Figure