Estimating Image Creativity

Nisan Chhetri, Christopher G. Healey, Sina BaghbanijamarXiv 2026 · arXiv

TL;DR

Researchers have developed a novel machine learning pipeline that decomposes the subjective concept of "image creativity" into three psychological dimensions: novelty, surprise, and arousal. By leveraging Vision-Language Models (VLMs) to capture semantic nuances that traditional neural networks miss, the framework produces creativity scores that are statistically indistinguishable from human judgments.

What Problem Does This Paper Address?

Quantifying creativity has long been a "holy grail" in both psychology and computer science. While we know a creative image when we see one, automating this judgment is notoriously difficult because creativity is inherently subjective and context-dependent.

Traditional computer vision tools, like Convolutional Neural Networks (CNNs), are excellent at identifying objects or textures but struggle with high-level semantic questions like, "Is this image unusual compared to everyday life?" The authors argue that to estimate creativity, we must first understand the underlying psychological dimensions that drive human perception—specifically novelty (how unique it is), surprise (how much it violates expectations), and arousal (the intensity of the emotional response).

Methods

The authors proposed a two-stage "image-to-creativity" pipeline:

  1. Dimension Estimation: The system first predicts the strength of an image's novelty, surprise, and arousal. The researchers compared three approaches for this task: standard CNNs (ResNet-18/50), Mean-Shift anomaly detection, and Vision-Language Models (VLMs).
  2. Creativity Prediction: These three dimension strengths are then fed into a final supervised learning model (testing Random Forest, SVC, KNN, and ANN) to produce a single creativity score on a scale of 0 to 7.
Figure 1. Proposed image to creativity score pipeline: ML models estimate an image’s creativity dimension strengths, which are then combined in a final ML model to return an image creativity score

A key innovation was the VLM prompting strategy. Instead of asking the model "How creative is this?", they used insights from human participant surveys to build prompts based on specific criteria, such as "Familiarity vs. Uniqueness" and "Plausibility vs. Impossibility."

Datasets

The study utilized three distinct image sets (45 images each, 135 total) to train and validate the models:

  • Novelty Set: Curated from Unsplash and Wikimedia Commons, spanning a range of "everyday" to "unique" scenes.
  • Surprise Set: Generated using Google’s Veo model to create physically impossible or highly unexpected scenes (e.g., a frog playing guitar).
  • Arousal Set: Sampled from the Open Affective Standardized Image Set (OASIS), a benchmark for emotional valence and arousal.

Ground truth data was collected via Amazon Mechanical Turk (MTurk), where "Master Turkers" provided ratings for both individual dimensions and overall creativity.

Key Insights and Ideas

  • Pixels vs. Semantics: CNNs performed poorly (25–65% accuracy) because they focus on statistical patterns in pixel values. Creativity, however, is a semantic judgment. VLMs succeeded because their pre-training on massive text-image corpora provides the "real-world knowledge" necessary to understand what is "normal" versus "novel."
  • The Power of Decomposition: By breaking creativity into dimensions, the researchers could build simpler, more interpretable models. This modularity also allows for easy expansion; new dimensions (like "value" or "elaboration") can be added to the pipeline without retraining the entire system.
  • Novelty is King: Ablation studies revealed that novelty is the primary driver of image creativity scores. When human input was used for novelty, the final creativity predictions were significantly more accurate than when using VLM-estimated novelty alone.

Key Results

The final creativity models achieved impressive parity with human raters. As shown in the table below, the relative accuracy ($acc_{rel}$) hovered around 77-78% regardless of the specific ML architecture used (RF, SVC, KNN, or ANN).

Table 1. acc_rel for RF, SVC, KNN, and ANN creativity score predictions using VLM-estimated strengths or participant-report strengths as input; t and p-values for VLM-based versus participant-based creativity scores.

Crucially, the $p$-values (ranging from 0.18 to 0.95) indicate that there was no statistically significant difference between creativity scores generated using VLM-estimated dimensions and those generated using actual human-reported dimensions. This suggests the VLM-based pipeline is a viable proxy for human judgment.

Limitations and Future Work

While promising, the study has a few notable limitations:

  • Sample Size: The dataset of 135 images is relatively small for deep learning standards.
  • Cultural Bias: The human raters were all US-based. Since creativity and arousal are culturally dependent, the model might not generalize to global audiences without further tuning.
  • Human Superiority in Novelty: The ablation studies showed that humans are still better at judging novelty than current VLMs. Improving the VLM's ability to grasp "unusualness" remains a challenge.

Future work aims to incorporate additional dimensions like "value" and explore how this "predict-then-combine" approach might apply to other subjective domains, such as evaluating the effectiveness of data visualizations.

Summary updated

Abstract

What makes an image creative? Although this may seem like a highly subjective decision, research in psychology suggests that creativity may be based on a set of fundamental creativity dimensions such as novelty, surprise, and arousal. A deeper understanding of image creativity can benefit both research and practical applications. Exploring how human viewers use creativity dimensions to evaluate image creativity may allow us to predict a target image's human-rated creativity automatically. We present a novel framework to estimate the perceived creativity of an image by: (1) predicting the strengths of an image's novelty, surprise, and arousal creativity dimensions, and (2) combining the creativity dimension strengths to estimate an image creativity score. Controlled human-subject experiments were conducted on Amazon Mechanical Turk to collect data to train supervised machine learning models to estimate an image's strengths on the three creativity dimensions. A final machine learning model takes the predicted dimension strengths as input to produce an estimated image creativity score. Results showed that our estimated creativity scores are statistically equivalent to human ratings of creativity for the images we tested. The novel contributions of our research are: (1) identification and validation of three psychological creativity dimensions used to estimate image creativity; (2) supervised machine learning models to automatically estimate novelty, surprise, and arousal strengths for an input image; (3) a supervised machine learning model to predict a creativity score from novelty, surprise, and arousal strengths for an input image, and (4) experimental methods to confirm the accuracy, advantages, and limitations of our approach.
BibTeX
@misc{chhetri2026estimating,
  author = {Nisan Chhetri and Christopher G. Healey and Sina Baghbanijam},
  title = {{Estimating Image Creativity}},
  year = {2026},
  eprint = {2611.15044},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15044},
}
Figure