The review of personalized image aesthetics assessment

Hancheng Zhu, Yong Zhou, Leida Li, Jiaqi Zhao, Wenliang DuJournal of Image and Graphics 2022 · DOI

Abstract

Image aesthetics assessment (IAA) is a rapidly evolving field, traditionally divided into generic and personalized approaches. While Generic Image Aesthetics Assessment (GIAA) focuses on universal aesthetic perceptions shared by the majority, Personalized Image Aesthetics Assessment (PIAA) targets the highly subjective preferences of individual users. Given that aesthetic experience is deeply personal, PIAA offers greater practical utility in real-world applications. This review outlines the current landscape of PIAA, categorizing existing models into collaborative filtering-based, user interaction-based, and aesthetic difference-based approaches. We analyze their design philosophies, strengths, and weaknesses, and discuss future directions in multi-modal fusion, explainability, and knowledge-driven modeling.

Introduction: From Generic to Personalized Beauty

For years, the computer vision community focused on Generic Image Aesthetics Assessment (GIAA). These models were trained to predict how a "typical" crowd would rate a photo based on established photographic principles like the rule of thirds, color harmony, and depth of field. However, beauty is notoriously subjective. A professional photographer’s criteria for a "good" shot differ vastly from those of a casual social media user.

Personalized Image Aesthetics Assessment (PIAA) addresses this subjectivity. The shift toward PIAA is driven by the need for AI that understands individual taste, which is influenced by culture, personality, and emotional state. However, moving from generic to personalized assessment introduces significant hurdles, most notably the Small Sample Problem—the difficulty of learning a user's complex preferences from only a few labeled examples—and the inherent difficulty of quantifying high-level subjective traits.

Methodological Frameworks

Current PIAA research can be grouped into three primary methodological "families":

  1. Collaborative Filtering (CF) Based Models: These models leverage the "wisdom of similar users." If User A and User B share similar tastes across a subset of images, the model predicts that they will likely agree on new images. By using similarity matrices, these systems can provide reasonable estimates even for users with limited personal data.
  2. User Interaction Based Models: These are active learning systems. They often employ ranking mechanisms or reinforcement learning where the user provides direct feedback (e.g., "I prefer photo A over B"). While highly accurate, these models face the challenge of "user fatigue," as most individuals are unwilling to provide extensive manual feedback.
  3. Aesthetic Difference Based Models: This approach treats personal taste as a deviation or "offset" from a generic aesthetic norm. The model first calculates a generic score and then learns a specific "difference model" to adjust that score according to the individual user’s unique bias. This is currently one of the most efficient methods for mobile deployment.

Datasets and the Data Scarcity Challenge

The development of PIAA is heavily dependent on specialized datasets that capture individual variations in rating. Key benchmarks include:

  • AADB (Aesthetics and Attributes Database): 10,000 images rated by 190 users across 11 aesthetic attributes.
  • FLICKR-AES: 40,000 images with ratings from 210 users, designed to highlight inter-user variance.
  • REAL-CUR: A dataset of 14 users' actual mobile phone albums, representing the "gold standard" for real-world application testing.
The challenge of limited data in personalized assessment. Diverse image types require robust models that can generalize from small user-specific samples.

The "Small Sample Problem" remains a central theme; as shown in the figure above, the diversity of image content makes it difficult for a model to learn a user's preference for every category without substantial data. Most modern PIAA approaches follow a two-stage training process: first, supervised training on massive generic datasets (like AVA) to learn basic features, followed by fine-tuning on tiny, user-specific datasets to capture individual nuances.

Key Insights and Future Directions

A major breakthrough in PIAA has been the move away from training models from scratch. By using GIAA models as a "prior" and fine-tuning them on small, user-specific datasets, researchers have significantly improved performance. Furthermore, the integration of "non-visual" data—such as a user's personality traits (the Big Five)—is showing promise in making aesthetic predictions more accurate.

Looking forward, the field is moving toward:

  • Multi-modal Fusion: Incorporating geolocation, captions, and even physiological signals (like heart rate) to better understand the context of an aesthetic judgment.
  • Explainability: Moving beyond "black box" predictions to models that can explain why a user might prefer a specific composition or color palette.
  • Cross-Cultural Diversity: Ensuring that PIAA models do not default to a Western-centric "monoculture" of beauty, but instead respect global aesthetic diversity.

Conclusion

Personalized Image Aesthetics Assessment represents a critical step toward more human-centric AI. By balancing generic aesthetic principles with individual nuances, PIAA is poised to revolutionize personalized recommendation systems, digital art tools, and automated photo curation.

Summary updated

BibTeX
@article{zhu2022review,
  author = {Hancheng Zhu and Yong Zhou and Leida Li and Jiaqi Zhao and Wenliang Du},
  title = {{The review of personalized image aesthetics assessment}},
  journal = {Journal of Image and Graphics},
  year = {2022},
  doi = {10.11834/jig.210211},
}
Figure