Introduction
As text-to-music (TTM) generation reaches new heights in audio fidelity, a critical question remains: does the music actually convey the intended emotion? While existing evaluations focus on technical quality or prompt alignment (e.g., "is there a piano?"), they often overlook the psychological impact on the listener. The AImoclips benchmark addresses this gap by quantifying emotion conveyance—the alignment between the emotional intent of a text prompt and the human perception of the resulting audio.
Methodology
The researchers developed a rigorous framework to evaluate six state-of-the-art models: four open-source (AudioLDM 2, MusicGen, Mustango, Stable Audio Open) and two commercial (Suno v4.5, Udio v1.5). The process involved four key stages:
- Emotion Selection: 12 emotion words were selected to represent the four quadrants of the Valence-Arousal (VA) model. Valence measures pleasure (positive vs. negative), while Arousal measures energy (high vs. low).
- Generation: The models generated 991 music clips (10 seconds each) based on these emotional prompts.
- Human Rating: 111 participants rated the clips on a 9-point scale for both valence and arousal.
- Analysis: The team established ground truth scores using the Warriner et al. (ANEW extension) dictionary to measure how far the AI-generated music deviated from the intended emotional weight.
Key Results and Insights
The "Neutrality Bias"
A primary finding of the study is the "centralizing tendency" or neutrality bias. Regardless of how extreme the emotional prompt was, models tended to produce music that listeners rated as more emotionally neutral. As shown in the quadrant analysis, high-valence prompts often resulted in lower-than-intended pleasure ratings, while low-valence prompts were perceived as less negative than intended.
Arousal vs. Valence
The study revealed that AI models are significantly better at conveying energy (arousal) than specific moods (valence). It is computationally simpler for a model to associate "energetic" with acoustic features like high tempo and volume than it is to distinguish the subtle musical nuances between "calm" (positive/low energy) and "gloomy" (negative/low energy).
Commercial vs. Open-Source Performance
A clear divide emerged between model types. Commercial models (Suno, Udio) exhibited a positive valence bias, often sounding more pleasant than intended. The authors suggest that the high production quality and "catchiness" of these models may inherently bias listeners toward positive feelings. Conversely, open-source models tended to lean toward the unpleasant (negative valence) side of the spectrum.
Conclusion and Future Work
AImoclips highlights a significant limitation in current TTM systems: a lack of "affective controllability." While models can follow instrumental instructions, they struggle to hit specific psychological targets reliably. Future work will focus on identifying the specific acoustic features (tempo, key, spectral brightness) that drive these emotional biases. The ultimate goal is to leverage the AImoclips dataset to fine-tune models, moving toward "emotion-to-music" systems capable of nuanced, intentional emotional expression.
Summary updated