Image aesthetics depends on context

Florian Simond, Nikolaos Arvanitopoulos, Sabine SüsstrunkICIP 2015 · OpenReview

Abstract

We investigate the influence of low-level image features for aesthetics prediction. We show that the aesthetic quality of a photography depends on its context. Image features learned from a specific image category are not necessarily the same as features learned from a generic image collection. Experiments conducted on specific image categories show that specific features obtain statistically significantly better results than generic ones.

Introduction

Does a "good" photo of a cat follow the same rules as a "good" photo of a city skyline? Traditional automated systems for assessing image aesthetics often treat all photos the same, applying a uniform set of features—such as color distribution or sharpness—across diverse databases. However, human perception of beauty is rarely universal across different subjects. For instance, a professional portrait might rely on a blurred background to emphasize a salient subject, whereas a high-quality landscape typically requires deep focus across the entire frame. This paper argues that image aesthetics are fundamentally context-dependent and that "generic" aesthetic models lose performance by ignoring semantic content.

Methodology

The researchers proposed a two-step pipeline to test the hypothesis that category-specific features improve aesthetic prediction.

Feature Extraction and Selection

The study defined 35 low-level aesthetic features grouped into three categories:

  • Global features: Brightness, color variance, and hue count.
  • Salient region features: Sharpness, size, and location of the most prominent part of the image.
  • Subject-Background relationship: Differences in color or sharpness between the main subject and the background.

Notably, the researchers improved upon standard "Rule of Thirds" calculations by measuring the distance of the salient region to "power points" (the intersections of the grid), rather than just averaging color values in the center. To identify the most effective features for each semantic category, they employed Sequential Forward Floating Search (SFFS). This iterative algorithm adds the most helpful features to a set while checking if removing existing ones improves overall accuracy. Classification was then performed using a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel.

Dataset

The study utilized a subset of the AVA (Aesthetic Visual Analysis) dataset, which contains over 250,000 images. To ensure reliable results, the authors selected 80,000 images representing the extremes of the distribution: the 40,000 highest-rated and 40,000 lowest-rated images. This approach focuses on clear aesthetic distinctions, avoiding the ambiguity of mediocre images. The analysis centered on four specific semantic categories: animals, nature, portraits, and city.

Examples of images from the AVA dataset with their aesthetic scores across different categories.

Key Insights

The analysis revealed several critical patterns in how features contribute to aesthetic quality:

  • Sharpness is Universal: Sharpness emerged as a top-tier predictor of aesthetic quality across all categories, from wildlife to architecture.
  • Saliency is Contextual: Features related to the "salient region" (the main subject) were highly ranked for animals and portraits, where a clear focal point is expected. Conversely, these features were less impactful for nature and city shots, which often feature complex, "busy" scenes without a single dominant object.
  • The Simplicity Factor: High-quality photos often exhibit lower hue counts and fewer edges, suggesting that "simple" images with clean color palettes and minimal visual clutter are generally perceived as more aesthetic.

Results

The researchers compared models trained on "generic" features against those trained on "category-specific" features. In every category tested, the specific features outperformed the generic ones:

Category Generic Features Accuracy Specific Features Accuracy
Animals 70.05% 70.35%
Nature 66.65% 67.12%
Portrait 68.34% 69.71%
City 71.63% 71.81%

Using a Wilcoxon signed-rank test, the authors confirmed that these improvements are statistically significant. The "Portrait" category showed the most substantial gain, reinforcing the idea that specific genres have unique aesthetic rules, such as subject-background separation, that generic models fail to capture fully.

Conclusion

This research demonstrates that aesthetic quality is not a one-size-fits-all metric. By narrowing the training data to specific categories, models can learn the nuanced "rules" of different photographic genres. While current methods require pre-existing semantic labels, future work could integrate deep learning classifiers to automate category identification. Furthermore, exploring finer sub-categories (e.g., macro vs. wide-angle nature shots) could further refine automated aesthetic assessment.

Summary updated

Abstract

We investigate the influence of low-level image features for aesthetics prediction. We show that the aesthetic quality of a photography depends on its context. Image features learned from a specific image category are not necessarily the same as features learned from a generic image collection. Experiments conducted on specific image categories show that specific features obtain statistically significantly better results than generic ones.
BibTeX
@inproceedings{simond2015image,
  author = {Florian Simond and Nikolaos Arvanitopoulos and Sabine Süsstrunk},
  title = {{Image aesthetics depends on context}},
  booktitle = {ICIP},
  year = {2015},
  url = {https://openreview.net/forum?id=BHl478jfGeWo},
}
Figure