TL;DR
Researchers have developed a novel machine learning pipeline that decomposes the subjective concept of "image creativity" into three psychological dimensions: novelty, surprise, and arousal. By leveraging Vision-Language Models (VLMs) to capture semantic nuances that traditional neural networks miss, the framework produces creativity scores that are statistically indistinguishable from human judgments.
What Problem Does This Paper Address?
Quantifying creativity has long been a "holy grail" in both psychology and computer science. While we know a creative image when we see one, automating this judgment is notoriously difficult because creativity is inherently subjective and context-dependent.
Traditional computer vision tools, like Convolutional Neural Networks (CNNs), are excellent at identifying objects or textures but struggle with high-level semantic questions like, "Is this image unusual compared to everyday life?" The authors argue that to estimate creativity, we must first understand the underlying psychological dimensions that drive human perception—specifically novelty (how unique it is), surprise (how much it violates expectations), and arousal (the intensity of the emotional response).
Methods
The authors proposed a two-stage "image-to-creativity" pipeline:
- Dimension Estimation: The system first predicts the strength of an image's novelty, surprise, and arousal. The researchers compared three approaches for this task: standard CNNs (ResNet-18/50), Mean-Shift anomaly detection, and Vision-Language Models (VLMs).
- Creativity Prediction: These three dimension strengths are then fed into a final supervised learning model (testing Random Forest, SVC, KNN, and ANN) to produce a single creativity score on a scale of 0 to 7.
A key innovation was the VLM prompting strategy. Instead of asking the model "How creative is this?", they used insights from human participant surveys to build prompts based on specific criteria, such as "Familiarity vs. Uniqueness" and "Plausibility vs. Impossibility."
Datasets
The study utilized three distinct image sets (45 images each, 135 total) to train and validate the models:
- Novelty Set: Curated from Unsplash and Wikimedia Commons, spanning a range of "everyday" to "unique" scenes.
- Surprise Set: Generated using Google’s Veo model to create physically impossible or highly unexpected scenes (e.g., a frog playing guitar).
- Arousal Set: Sampled from the Open Affective Standardized Image Set (OASIS), a benchmark for emotional valence and arousal.
Ground truth data was collected via Amazon Mechanical Turk (MTurk), where "Master Turkers" provided ratings for both individual dimensions and overall creativity.
Key Insights and Ideas
- Pixels vs. Semantics: CNNs performed poorly (25–65% accuracy) because they focus on statistical patterns in pixel values. Creativity, however, is a semantic judgment. VLMs succeeded because their pre-training on massive text-image corpora provides the "real-world knowledge" necessary to understand what is "normal" versus "novel."
- The Power of Decomposition: By breaking creativity into dimensions, the researchers could build simpler, more interpretable models. This modularity also allows for easy expansion; new dimensions (like "value" or "elaboration") can be added to the pipeline without retraining the entire system.
- Novelty is King: Ablation studies revealed that novelty is the primary driver of image creativity scores. When human input was used for novelty, the final creativity predictions were significantly more accurate than when using VLM-estimated novelty alone.
Key Results
The final creativity models achieved impressive parity with human raters. As shown in the table below, the relative accuracy ($acc_{rel}$) hovered around 77-78% regardless of the specific ML architecture used (RF, SVC, KNN, or ANN).
Crucially, the $p$-values (ranging from 0.18 to 0.95) indicate that there was no statistically significant difference between creativity scores generated using VLM-estimated dimensions and those generated using actual human-reported dimensions. This suggests the VLM-based pipeline is a viable proxy for human judgment.
Limitations and Future Work
While promising, the study has a few notable limitations:
- Sample Size: The dataset of 135 images is relatively small for deep learning standards.
- Cultural Bias: The human raters were all US-based. Since creativity and arousal are culturally dependent, the model might not generalize to global audiences without further tuning.
- Human Superiority in Novelty: The ablation studies showed that humans are still better at judging novelty than current VLMs. Improving the VLM's ability to grasp "unusualness" remains a challenge.
Future work aims to incorporate additional dimensions like "value" and explore how this "predict-then-combine" approach might apply to other subjective domains, such as evaluating the effectiveness of data visualizations.
Summary updated