Multi-Level Attribute-Guided-Based Adaptive Multi-Dilated Convolutional Network for Image Aesthetic Assessment

Sumei Li, Mingxuan Xie, Wei XiangJournal of Imaging 2025 · DOI

TL;DR

This paper introduces MAADN, a novel deep learning model for image aesthetic assessment that addresses two key limitations: the lack of hierarchical attribute guidance and the distortion of image aspect ratios during preprocessing. By employing a multi-level attribute-guided approach and an adaptive dilated convolution module, MAADN achieves state-of-the-art performance and better preserves image composition for more accurate aesthetic evaluation.

What Problem Does This Paper Address?

Evaluating the aesthetic quality of an image is a complex task, crucial for applications like image enhancement and recommendation systems. Existing deep learning models for Image Aesthetic Assessment (IAA) often struggle with two main issues:

  1. Ignoring Hierarchical Attribute Interactions: Human aesthetic judgment is a progressive process, considering features from low-level details to high-level semantics. Many IAA models fail to capture the hierarchical relationships between various visual attributes (like color, composition, lighting) and the overall aesthetic score. They often fuse attribute information at a single, late stage, missing out on richer, multi-level interactions.
  2. Preprocessing Distortion: Standard Convolutional Neural Networks (CNNs) require fixed-size inputs, forcing images to be resized or cropped. This preprocessing can distort the original spatial composition and structural integrity of an image, leading to a loss of aesthetic information and potentially inaccurate assessments.

This paper aims to overcome these limitations by proposing a model that leverages multi-level attribute guidance and an adaptive approach to convolution that preserves image structure.

Methods

The proposed model, MAADN (Multi-level Attribute-Guided Adaptive Multi-Dilated Convolutional Network), features a dual-branch architecture:

  • Attribute Branch: This branch, based on a pre-trained ResNet50, extracts multi-level attribute features from the input image. These attributes include elements like balancing, color harmony, content, depth of field, and more.
  • Aesthetic Branch: This branch is designed to learn aesthetic features. It incorporates the novel Adaptive Multi-Dilate Rate Convolution Module (AMDM). AMDM addresses the preprocessing issue by using parallel dilated convolution kernels with different dilation rates. Crucially, it adaptively learns weights to fuse these multi-scale features, preserving the original image composition and mitigating information loss caused by resizing. This is inspired by the observation that image aspect ratios in datasets like AVA are predominantly within a certain range, allowing for tailored dilation rates.
  • Attention-based Attribute-Guided Aesthetic Module (AGAM): This module acts as the bridge between the two branches. Inspired by human visual perception, AGAM uses channel and spatial attention mechanisms. It first uses channel attention to capture global image perception (like color and light) and then spatial attention to focus on local details. This allows attribute features to effectively guide and enhance the aesthetic features extracted by the Aesthetic Branch.

The overall architecture simulates a hierarchical human visual system, where attribute features guide aesthetic feature extraction at multiple levels.

Datasets

The paper evaluates MAADN on three publicly available datasets:

  • AVA (Aesthetic Visual Analysis): The largest dataset used, containing over 250,000 images with aesthetic score distributions (1-10), semantic content, and photographic style annotations. The study uses a subset of 229,937 images for training.
  • AADB (Aesthetic Attributes Database): Contains 10,000 images with overall aesthetic scores (1-5) and 11 different aesthetic attribute scores. This dataset is used for pre-training the Attribute Branch.
  • PARA: Contains 31,220 images with aesthetic scores (1-5), aesthetic attribute scores, emotion categories, and scene categories.

Key Insights and Ideas

  • Hierarchical Guidance is Crucial: The paper strongly advocates for and demonstrates the benefit of guiding aesthetic feature learning with attribute features at multiple levels of the network, mimicking human visual processing. This multi-level approach significantly outperforms single-level guidance.
  • Adaptive Dilated Convolutions Preserve Structure: The AMDM module is a key innovation. By adaptively weighting features from dilated convolutions with different rates, it effectively mitigates the negative impact of image resizing/cropping, preserving crucial compositional information that traditional CNNs lose.
  • Attention Enhances Attribute-Aesthetic Interaction: AGAM's use of channel and spatial attention effectively refines the interaction between attribute and aesthetic features, leading to improved accuracy and interpretability.
  • Synergistic Integration: The combination of multi-level guidance, AGAM, and AMDM in the full MAADN model yields the best performance, highlighting the synergistic benefits of these components.

Key Results

  • State-of-the-Art Performance: MAADN achieves state-of-the-art results on the AVA dataset for SRCC (0.714) and PLCC (0.728), and competitive results for ACC (81.94%) and EMD (0.043).
  • Broad Adaptability: The model also demonstrates strong performance on the AADB and PARA datasets, achieving the best SRCC and PLCC on both.
  • Ablation Studies Confirm Effectiveness:
    • Multi-level guidance significantly outperforms single-level guidance.
  • AGAM improves performance by enhancing attribute-aesthetic interactions.
  • AMDM consistently boosts performance across various configurations, especially in mitigating preprocessing damage.
  • Visualizations Show Precise Localization: Grad-CAM visualizations demonstrate that MAADN accurately focuses on aesthetically critical regions, aligning well with human perceptual focus, even in challenging scenarios like monochrome images or occluded scenes.

Limitations and Future Work

  • Computational Cost: The AMDM module, while effective, introduces a significant increase in model complexity and parameters, making it computationally intensive.
  • Attribute Representation Reliability: The AGAM module might face limitations when attribute representations are unreliable or when the domain shifts significantly from the training data.

For future work, the authors suggest exploring more efficient architectural designs to reduce computational complexity and investigating optimization strategies to enhance the model's robustness across diverse aesthetic domains. They also plan to further explore architectural designs that can reduce computational complexity and improve robustness across various aesthetic domains.

Summary updated

Abstract

Image aesthetic assessment (IAA) is crucial for both scientific research and practical applications, and numerous studies have achieved promising performance. However, they still exhibit two major limitations: the neglect of hierarchical interactions between attribute features and aesthetic features, and the distortion of the original aspect ratio during image preprocessing, which leads to a loss of aesthetic information. To address these issues, we propose a Multi-level Attribute-Guided Adaptive Multi-Dilated Convolutional Network (MAADN), which leverages multi-level attribute features to guide aesthetic assessment and reduces the negative impact of image preprocessing through adaptive dilated convolution. Specifically, we employ a dual-branch architecture: one branch extracts multi-level attribute features, while the other learns aesthetic features under the guidance of these attributes. We further design an Attention-based Attribute-Guided Aesthetic Module (AGAM), which utilizes visual attention mechanisms to enhance the guidance of attributes. Additionally, we design an Adaptive Multi-Dilate Rate Convolution Module (AMDM) that generates weights adaptively through the network to fuse dilated convolution features with different dilation rates, rather than simply calculating weights based on aspect ratios. This approach effectively alleviates the negative effects of image preprocessing while maintaining training flexibility. Extensive experimental results demonstrate that the proposed model outperforms current state-of-the-art approaches. Furthermore, visual analysis confirms MAADN's precise localization capability for aesthetically critical regions.
BibTeX
@article{li2025multi,
  author = {Sumei Li and Mingxuan Xie and Wei Xiang},
  title = {{Multi-Level Attribute-Guided-Based Adaptive Multi-Dilated Convolutional Network for Image Aesthetic Assessment}},
  journal = {Journal of Imaging},
  year = {2025},
  doi = {10.3390/jimaging11120420},
}
Figure