Abstract
Now many mobile phones embed deep-learning models for evaluation or guidance on photography. These models cannot provide detailed results like human pose scores or scene color scores because of the rare of corresponding aesthetic attribute data. However, the annotation of image aesthetic attribute scores requires experienced artists and professional photographers, which hinders the collection of large-scale fully-annotated datasets. In this paper, we propose to replace image attribute labels with feature extractors. First, a novel aesthetic attribute evaluation framework based on attribute features is proposed to predict attribute scores and overall scores. We call it the F2S (attribute features to attribute scores) model. We use networks from different tasks to provide attribute features to our F2S models. Then, we define an aesthetic attribute contribution to describe the role of aesthetic attributes throughout an image and use it with the attribute scores and the overall scores to train our F2S model. Sufficient experiments on publicly available datasets demonstrate that our F2S model achieves comparable performance with those trained on the datasets with fully-annotated aesthetic attribute score labels. Our method makes it feasible to learn meaningful attribute scores for various aesthetic attribute sets in different types of images with only overall aesthetic scores.
Introduction
In the field of Image Aesthetic Assessment (IAA), the goal has shifted from merely predicting a binary "good/bad" label to providing nuanced, interpretable feedback. Users want to understand why a photograph is considered aesthetically pleasing—whether it is due to excellent lighting, a strong adherence to the rule of thirds, or compelling subject matter. However, training models to provide this level of detail is historically difficult. High-quality datasets with granular attribute-level labels (e.g., "Color Harmony: 8/10") are scarce because they require annotation by professional photographers, making large-scale data collection prohibitively expensive.
Most existing datasets provide only a single "overall" aesthetic score. This paper addresses this gap by introducing the F2S (Attribute Features to Attribute Scores) framework. The core question is: Can we learn to predict individual aesthetic attributes using only the overall score as a supervisory signal?
Methodology: The F2S Framework
The F2S framework treats aesthetic attributes as a combination of specialized features and a learnable "contribution" weight. It operates through three main components:
1. Attribute Feature Extraction
Rather than learning aesthetic features from scratch, the model leverages pre-trained networks from diverse computer vision tasks to act as proxies for specific attributes. This "cross-task synergy" assumes that models trained for objective tasks already possess the necessary "vocabulary" to judge subjective aesthetics. For instance:
- Content & Semantic: Leverages MaskFormer for scene segmentation.
- Object Emphasis: Uses DETR for object detection.
- Rule of Thirds (RoT): Employs a context-aware line detector (DHT).
- Depth of Field (DoF): Utilizes Monodepth2 for depth estimation.
2. The Contribution Module
Aesthetics are contextual; not every attribute is equally important in every image. The F2S model uses an attention-based module to calculate Attribute Contributions ($C_i$). In a portrait, "face lighting" and "pose" might be the primary drivers of the score, whereas in a landscape, "vivid color" and "composition" might take precedence.
3. Weakly Supervised Learning
The model is trained using a composite loss function. It predicts individual attribute scores ($S_i$) and combines them with their respective contribution weights ($C_i$) to estimate the overall score. By optimizing the model to match the ground truth overall score, the signal is effectively "back-propagated" into the individual attribute branches, allowing them to learn meaningful scores without direct attribute labels.
Experimental Evaluation
Datasets
The researchers evaluated F2S across several major benchmarks, including AVA (255k+ images), AADB (10k images with 11 attributes), EVA (focusing on visual/compositional quality), and PCCD. They also curated AVA-Portraits, a subset of 4,500 images, to test specialized attributes like "face lighting" and "emotion."
Performance and Interpretability
On the AADB dataset, the weakly supervised F2S model achieved a Spearman’s Rank Correlation Coefficient (SRCC) of 0.728, which is remarkably close to the 0.730 achieved by a fully supervised model trained with explicit attribute labels. This demonstrates that the overall score contains sufficient information to guide the learning of individual attributes when paired with the right feature extractors.
As shown in the figure above, the model provides a transparent breakdown of its reasoning. For example, in the PCCD dataset (third row), the "Composition & Placement" (C&P) and "Depth of Field" (DoF) attributes are identified as major contributors to the overall score. The inclusion of an "Extra" attribute branch ensures that aesthetic factors not covered by the specific modules are still accounted for, preventing the model from force-fitting quality into a limited set of definitions.
Discussion and Limitations
While the F2S framework significantly reduces the need for expert labels, it introduces a new requirement: the expert selection of feature extractors. A human must still decide which pre-trained model is appropriate for a given aesthetic attribute. Furthermore, the system's accuracy is inherently capped by the performance of these underlying models; for instance, an inaccurate depth estimator will lead to an unreliable Depth of Field score.
Conclusion
The F2S model demonstrates that it is feasible to learn detailed, meaningful aesthetic attribute scores using only overall labels. By bridging the gap between objective computer vision tasks and subjective aesthetic assessment, this framework provides a scalable way to build interpretable AI for photography, offering users not just a score, but a reason.
Summary updated