Understanding and Mapping Natural Beauty

Scott Workman, Richard Souvenir, Nathan Jacobs

Abstract: While natural beauty is often considered a subjective property of images, in this paper, we take an objective approach and provide methods for quantifying and predicting the scenicness of an image. Using a dataset containing hundreds of thousands of outdoor images captured throughout Great Britain with crowdsourced ratings of natural beauty, we propose an approach to predict scenicness which explicitly accounts for the variance of human ratings. We demonstrate that quantitative measures of scenicness can benefit semantic image understanding, content-aware image processing, and a novel application of cross-view mapping, where the sparsity of ground-level images can be addressed by incorporating unlabeled overhead images in the training and prediction steps. For each application, our methods for scenicness prediction result in quantitative and qualitative improvements over baseline approaches.

Introduction

Quantifying the aesthetic value of a landscape has long been considered a purely subjective endeavor. However, recent research suggests that "scenicness"—the natural beauty of outdoor scenes—exhibits a surprising degree of consensus among observers. This paper presents a deep learning framework designed to quantify and map scenicness across Great Britain. By training on a massive crowdsourced dataset and explicitly modeling the variance in human judgment, we demonstrate that natural beauty can be treated as an objective, measurable property.

Visual contrast between high-scenicness natural landscapes and low-scenicness industrial infrastructure.

Methodology

The core of our approach is a Deep Convolutional Neural Network (CNN) based on the GoogLeNet architecture, pre-trained on the Places dataset. A key technical challenge is moving beyond simple mean ratings to account for the inherent uncertainty and variance in human perception.

We evaluated three loss functions for training:

  • AVERAGE: A baseline predicting the mean rating.
  • DISTRIBUTION: Cross-entropy on the normalized rating distribution.
  • MULTINOMIAL: Maximizing multinomial log-likelihood.

The MULTINOMIAL approach proved most effective, as it treats ratings as samples from a distribution, thereby capturing the range of human opinion more accurately than a single average score.

To address the spatial sparsity of ground-level photography, we developed a hybrid cross-view model. This model integrates overhead satellite imagery with features from the $k$-nearest ground-level images to estimate scenicness at any given coordinate, enabling the creation of dense, high-resolution beauty maps.

Datasets and Human Perception

Our research utilizes two primary datasets:

  1. ScenicOrNot (SoN): Comprising approximately 185,000 images from Great Britain, each with at least five crowdsourced ratings on a 1–10 scale.
  2. Cross-View ScenicOrNot (CVSoN): A novel dataset pairing SoN ground-level images with co-located overhead orthorectified imagery from Bing Maps.

Analysis of these datasets confirms that humans generally agree on scenic value. Mountainous terrain and water features consistently receive high ratings, while industrial structures like power lines correlate with lower scores.

Examples of scenic and non-scenic images from the ScenicOrNot dataset with their corresponding human rating distributions.

Key Results

  • Distribution Matching: The MULTINOMIAL model matched the human rating distribution for 58.4% of test images (Kolmogorov-Smirnov test), a significant improvement over the 14.8% achieved by the AVERAGE baseline.
  • Mapping Accuracy: The hybrid cross-view approach achieved an AUC of 68.51% for binary scenicness classification, outperforming standard 1-Nearest Neighbor interpolation (64.38%).
  • Content-Aware Processing: We demonstrated the model's utility in automated image cropping. By identifying "maximally scenic" sub-regions, the model can automatically improve aesthetic composition, such as cropping out a foreground parking lot to focus on a distant mountain range.

Discussion and Future Work

While our models successfully quantify scenicness in Great Britain, several avenues for future research remain. The current models are geographically biased toward the UK; cultural perceptions of beauty may differ in other ecological contexts like deserts or rainforests. Additionally, transient factors such as weather, season, and time of day—which significantly impact perceived beauty—are not yet explicitly modeled. Future efforts will focus on integrating these transient attributes and expanding the dataset to a global scale to explore the potential for a universal model of scenicness.

Summary updated

Abstract

While natural beauty is often considered a subjective property of images, in this paper, we take an objective approach and provide methods for quantifying and predicting the scenicness of an image. Using a dataset containing hundreds of thousands of outdoor images captured throughout Great Britain with crowdsourced ratings of natural beauty, we propose an approach to predict scenicness which explicitly accounts for the variance of human ratings. We demonstrate that quantitative measures of scenicness can benefit semantic image understanding, content-aware image processing, and a novel application of cross-view mapping, where the sparsity of ground-level images can be addressed by incorporating unlabeled overhead images in the training and prediction steps. For each application, our methods for scenicness prediction result in quantitative and qualitative improvements over baseline approaches.
BibTeX
@misc{workman2016understanding,
  author = {Scott Workman and Richard Souvenir and Nathan Jacobs},
  title = {{Understanding and Mapping Natural Beauty}},
  year = {2016},
  eprint = {1612.03142},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1612.03142},
}
Figure