Aesthetic Attribute Assessment of Images Numerically on Mixed Multi-attribute Datasets

Xin Jin, Xinning Li, Hao Lou, Chenyu Fan, Qiang Deng, Chaoen Xiao, Shuai Cui, Amit Kumar Singh

Abstract

Traditional Image Aesthetic Assessment (IAA) typically relies on a single scalar value to represent image quality, failing to capture the multidimensional nature of human perception. This paper introduces a multitasking network designed to evaluate specific aesthetic attributes—such as color, light, and composition—numerically. By combining a new mixed dataset (AMD-A) with a novel architecture that fuses deep learning features with handcrafted photography rules, the proposed method achieves superior precision and interpretability compared to traditional single-score models.

Introduction

Image aesthetics are inherently subjective and complex. While deep learning has significantly advanced automated scoring, most models provide a single "black box" score that lacks transparency. A single number cannot explain why an image is considered high-quality; for instance, a photo might possess vibrant lighting but suffer from poor composition.

The authors identify three primary challenges in current IAA research: the lack of fine-grained detail in scoring, the limitations of existing datasets (which often have subjective or sparse annotations), and the computational complexity associated with multi-attribute assessment. This research addresses these gaps by proposing an efficient multitasking network that leverages both deep features and classical photography principles.

Methodology

Network Architecture

The researchers utilized EfficientNet-B0 as the core backbone due to its high performance-to-parameter ratio. To enhance feature extraction, Efficient Channel Attention (ECA) modules were integrated between the backbone and the attribute-specific sub-networks. This allows the model to adaptively select important feature channels without the dimensionality reduction that often leads to information loss.

Feature Fusion with External Attributes

A key innovation of this work is the fusion of "handcrafted" photography features with deep learning features. Instead of relying solely on the neural network to learn aesthetic rules, the authors explicitly calculate 21 external attributes:

  • Light Features (4): Mean and standard deviation of L and V channels.
  • Color Features (7): RGB/HSV dominant colors, contrast ratios, and hue histograms.
  • Composition Features (10): Numerical proximity to the golden section, rule of thirds, symmetry, and geometric shapes (triangles, circles).

These features are fused into the attribute sub-networks, providing the model with a grounded understanding of classical photography rules.

Teacher-Student Soft Loss

To improve the stability of fine-grained score prediction, the authors implemented a "Soft Loss" function inspired by knowledge distillation. A coarse-grained classification sub-network acts as a "teacher," guiding the fine-grained regression sub-network. This helps the model learn the general aesthetic "neighborhood" of an image before refining the exact decimal value.

Dataset: AMD-A

The authors introduced the Aesthetic Mixed Dataset with Attributes (AMD-A), a benchmark containing 16,924 images. This dataset aggregates and filters data from several existing sources—including EVA, AADB, and PCCD—alongside self-built sets (PADB, HADB). This diverse collection ensures a robust distribution of labels for training and evaluation.

Experimental Results

The proposed method demonstrated significant improvements over the baseline EfficientNet-B0 model:

  • Attribute Accuracy: The inclusion of handcrafted features boosted the Spearman Rankorder Correlation Coefficient (SROCC) across all categories. Notably, the SROCC for the Light attribute rose from 0.6939 to 0.7126.
  • Overall Scoring: The Teacher-Student Soft Loss reduced the Mean Square Error (MSE) from 0.0129 to 0.0113 and increased the overall SROCC from 0.8424 to 0.8604.
  • Interpretability: Qualitative results show the model can provide a detailed breakdown of scores (e.g., Color: 0.77, Light: 0.80, Composition: 0.78), allowing users to understand the specific drivers behind an image's aesthetic rating.

Conclusion

This research demonstrates that handcrafted photography rules still hold significant value in the era of deep learning. By explicitly feeding these rules into a multitasking network and using a guided loss function, the authors have created a model that is not only more accurate but also more interpretable. Future work may involve expanding the model to capture more complex semantic attributes, such as storytelling or emotional impact.

Summary updated

BibTeX
@article{jin2022aesthetic,
  author = {Xin Jin and Xinning Li and Hao Lou and Chenyu Fan and Qiang Deng and Chaoen Xiao and Shuai Cui and Amit Kumar Singh},
  title = {{Aesthetic Attribute Assessment of Images Numerically on Mixed Multi-attribute Datasets}},
  journal = {ACM Trans. Multim. Comput. Commun. Appl.},
  year = {2022},
  doi = {10.1145/3547144},
}
Figure