Introduction
The intersection of human emotion and sensory perception has long been a subject of psychological inquiry, typically relying on subjective self-reporting. However, recent advancements in non-linear dynamics suggest that emotional stimuli—whether auditory or visual—possess inherent structural complexities that can be quantified. This study explores a novel mathematical framework using fractal analysis to objectively classify emotions in music and art, specifically focusing on the cross-modal interaction between these two domains.
Methodology
The research utilizes a suite of non-linear data analysis tools to extract mathematical signatures from emotional stimuli. The primary metric employed is Detrended Fluctuation Analysis (DFA). For 1D audio signals, DFA calculates the scaling exponent (Hurst exponent, $\alpha$), which quantifies long-range temporal correlations. To analyze the visual domain, the authors propose a 2D-DFA algorithm that processes paintings by decomposing them into Red, Green, and Blue (RGB) components to calculate spatial scaling exponents.
Additionally, Detrended Cross-Correlation Analysis (DCCA) was used to measure the structural similarity between different signals within and across emotional categories. The study utilized 12 validated stimuli: six audio clips from the International Affective Digitized Sounds (IADS) database and six famous paintings (e.g., works by Van Gogh and Picasso), evenly split between "Happy" (positive valence) and "Sad" (negative valence) categories.
Results and Discussion
Fractal Signatures of Emotion
A central finding of the study is that "sad" stimuli exhibit significantly higher long-range correlations than "happy" ones. In both the audio and visual domains, a DFA scaling exponent greater than 1.5 consistently identified stimuli belonging to the sad category.
As shown in the figure above, audio clips 4-6 (Sad) show higher exponents (~1.8-1.9) compared to clips 1-3 (Happy, ~1.5-1.6). In the visual domain, the scaling exponents for sad paintings were particularly pronounced in the green and blue color channels, providing a mathematical basis for the artistic intuition that "cool" colors evoke melancholy.
Cross-Modal Consistency and Correlation
The study revealed a remarkable consistency in how emotions are mathematically structured across different senses. The Pearson correlation coefficient between the DFA values of happy audio and happy images reached 0.987, while sad stimuli showed a correlation of 0.96. This suggests that the "mathematics of feeling" may be universal, transcending the specific sensory modality.
DCCA results further supported these findings. Cross-correlation was strongest among stimuli of the same valence, particularly within the sad category. Conversely, inter-valence comparisons (comparing a happy stimulus to a sad one) often resulted in coefficients near zero, indicating a lack of shared structural complexity between opposing emotional states.
Conclusion
By moving beyond subjective surveys toward computational classification, this research demonstrates that fractal geometry can serve as a robust proxy for emotional valence. The discovery of a consistent mathematical signature across audio and visual domains opens new avenues for automated emotion recognition systems, adaptive media, and therapeutic interventions. Future work will aim to expand the dataset and explore whether these fractal tools can differentiate between more nuanced emotional states beyond the binary of happy and sad.
Summary updated