Abstract
Learning computational models of image aesthetics can have a substantial impact on visual art and graphic design. Although automatic image aesthetics assessment is a challenging topic by its subjective nature, psychological studies have confirmed a strong correlation between image layouts and perceived image quality. While previous state-of-the-art methods attempt to learn holistic information using deep Convolutional Neural Networks (CNNs), our approach is motivated by the fact that Graph Convolutional Network (GCN) architecture is conceivably more suited for modeling complex relations among image regions than vanilla convolutional layers. Specifically, we present a Hierarchical Layout-Aware Graph Convolutional Network (HLA-GCN) to capture layout information. It is a dedicated double-subnet neural network consisting of two LA-GCN modules. The first LA-GCN module constructs an aesthetics-related graph in the coordinate space and performs reasoning over spatial nodes. The second LA-GCN module performs graph reasoning after aggregating significant regions in a latent space. The model output is a hierarchical representation with layout-aware features from both spatial and aggregated nodes for unified aesthetics assessment. Extensive evaluations show that our proposed model outperforms the state-of-the-art on the AVA and AADB datasets across three different tasks.
Introduction
Assessing the aesthetic quality of an image is a complex task due to its inherent subjectivity. However, principles from photography and psychology suggest that layout and composition—such as the rule-of-thirds, balance, and symmetry—are objective indicators of visual appeal. Traditional Deep Convolutional Neural Networks (CNNs) struggle to capture these elements for two primary reasons. First, they typically require fixed-size square inputs, necessitating cropping or warping that destroys the original composition. Second, their local receptive fields are ill-suited for modeling long-range dependencies between distant image regions, which is essential for understanding global balance.
The HLA-GCN Framework
To address these limitations, we propose the Hierarchical Layout-Aware Graph Convolutional Network (HLA-GCN). This framework treats aesthetics assessment as a graph reasoning problem rather than a simple pixel-classification task. The architecture consists of two primary subnets designed to capture layout information at different levels of abstraction.
Layout-Aware Graph Reasoning
A core innovation of our method is the Layout-Preserving Warping and the construction of the aesthetics-related graph. Instead of ignoring the distortion caused by resizing, we calculate the original aspect ratio and embed it directly into the network.
In the Coordinate Space (1st LA-GCN), the image is represented as a grid of spatial nodes. We build a graph where edge weights are determined by a combination of content similarity and aspect-ratio-embedded spatial distance. This allows the model to perform dynamic convolutions that link distant but visually related elements.
In the Latent Space (2nd LA-GCN), the model aggregates similar spatial nodes into high-level semantic components (e.g., grouping all regions belonging to the "sky"). Reasoning on these aggregated nodes allows the model to capture the broader semantic layout without the computational overhead of a fully connected spatial graph.
Experimental Results
The HLA-GCN was evaluated on the AVA (Aesthetic Visual Analysis) and AADB (Aesthetics and Attributes Database) datasets. Our model achieved state-of-the-art (SOTA) performance across three tasks: mean score prediction, binary classification, and rating distribution prediction.
- AVA Dataset: The model achieved 84.6% accuracy in binary classification and a Spearman’s Rank Correlation Coefficient (SRCC) of 0.665, significantly outperforming baseline models like NIMA.
- AADB Dataset: The model reached an SRCC of 0.899, demonstrating its robustness across different dataset scales.
- Efficiency: When implemented using the Jittor framework, the model achieved an inference speed of 3.62 ms per image, making it viable for real-time applications.
Conclusion
By shifting from grid-based convolutions to layout-aware graph reasoning, the HLA-GCN effectively captures the structural nuances of image composition. While the model excels at structural assessment, future work will focus on integrating higher-level semantic understanding to better capture the emotional and narrative aspects of image aesthetics.
Summary updated