The Relationship Between Emotion and Music Production Quality

Dm Ronan, Julian Reiss, Hatice GüneşQueen 2016

TL;DR

This study explores how music production quality impacts listeners' emotional responses, both induced and perceived. It finds that higher production quality significantly influences critical listeners' emotions, suggesting that skilled mixing matters more to those with trained ears.

What Problem Does This Paper Address?

The music production process involves several stages, with mixing being crucial for shaping the final sound and conveying intended emotions. While the importance of emotion in music is well-established, the direct link between the quality of a music mix and a listener's emotional experience remains underexplored. Specifically, this paper investigates whether variations in music production quality (e.g., a "high quality" vs. "low quality" mix) lead to discernible differences in how listeners feel emotions (induced) and how they perceive emotions within the music. This is particularly relevant for understanding the value of skilled audio engineers and for developing automated mixing systems.

Methods

The researchers employed a multi-modal approach to capture listeners' emotional responses. This included:

  • Self-Report: Participants rated their induced emotions using the Geneva Emotion Music Scale (GEMS-9) and perceived emotions on Arousal-Valence-Tension (AVT) scales. They also rated their liking of each mix.
  • Physiological Measures: Skin conductance response (SCR) was recorded to measure arousal.
  • Facial Expression Analysis: The Facial Action Coding System (FACS) was used to track individual facial muscle movements (Action Units or AUs) associated with emotional expression.
  • Head Nod/Shake Detection: Video analysis was used to detect head nods and shakes, which can indicate positive or negative responses, respectively.

The study design involved comparing listeners' responses to high-quality and low-quality mixes of the same songs. Participants were categorized into "critical listeners" (with music production experience) and "non-critical listeners" (without). Statistical analyses, including Wilcoxon Signed Rank and Mann-Whitney U tests, were used to compare responses across different conditions and listener groups.

Datasets

The study utilized ten songs, each with multiple mixes. The specific mixes were sourced from a previous experiment ([2], [3]) where they had been rated for quality. For this study, the lowest and highest quality mixes of each song were selected. Some song titles were omitted due to copyright restrictions.

Key Insights and Ideas

  • Critical Ears, Deeper Emotions: The most striking finding is that music production quality has a significantly greater emotional impact on critical listeners compared to non-critical listeners. This suggests that a well-crafted mix is more likely to evoke distinct emotional responses in those with trained auditory perception.
  • High Quality Matters Most for Critical Listeners: The difference in emotional response between high and low quality mixes was more pronounced for critical listeners, particularly for emotions like joyful activation and sadness. This implies that subtle mixing nuances are better perceived and acted upon by experienced listeners.
  • Head Nods and Genre Preference: Non-critical listeners tended to nod their heads more to low-quality mixes, suggesting they might enjoy the music regardless of technical quality. This behavior, along with genre preferences, was noted as a potential factor influencing emotional responses.
  • Facial Expressions as Indicators: While GSR results were largely inconclusive, facial action unit analysis and head nod/shake detection showed promise in differentiating emotional responses, especially when comparing critical and non-critical listeners to high-quality mixes.

Key Results

  • GEMS-9: Critical listeners showed significantly different distributions of induced emotions between the two mix types, while non-critical listeners did not. High-quality mixes led to greater differences in emotional distribution between listener types.
  • Arousal-Valence-Tension (AVT): Critical listeners exhibited more significant differences in perceived emotions, particularly in arousal and tension, when comparing mixes.
  • Head Nod/Shake: Significant differences were observed, with non-critical listeners nodding more to low-quality mixes.
  • Facial Action Units (AUs): High-quality mixes showed a greater effect on AU distribution between listener types, with specific AUs (like inner brow raiser and brow lowering) showing significant differences.

Limitations and Future Work

  • Sample Size and Genre Diversity: The authors acknowledge that using a wider variety of song genres would have been beneficial, as some participants expressed dissatisfaction with the chosen songs. A larger sample size would also improve generalizability.
  • Confounding Factors: The influence of lyrics on emotional response was noted as a potential confounding variable, especially for non-native English speakers.
  • Training Effects: While training was provided to familiarize participants with the interface, the authors ponder if more extensive training for non-critical listeners could blur the distinction between critical and non-critical groups.
  • Future Directions: The paper suggests future work could involve pair-wise ranking of mixes (instead of Likert scales) for more nuanced affect studies, tracking singing along as a measure of engagement, and investigating the impact of training on non-critical listeners. Further research into automatic mixing systems and the correlation of emotional measures with audio features is also recommended.

Summary updated

Abstract

It is commonly known that music expresses emotion. In music production, the role of the mix engineer is to take a piece of recorded music and convey the emotions expressed as professionally sounding as possible. In this work, we investigated the relationship between music production quality and musically induced and perceived emotions.
BibTeX
@inproceedings{ronan2016relationship,
  author = {Dm Ronan and Julian Reiss and Hatice Güneş},
  title = {{The Relationship Between Emotion and Music Production Quality}},
  booktitle = {Queen},
  year = {2016},
}
Figure