TL;DR
This study investigates how individual differences in how we perceive sound – specifically, whether we focus on the fundamental frequency (FO) or higher harmonics (spectral) – relate to our ability to identify emotions in speech. The researchers found that individuals with a spectral listening preference were better at identifying emotions, suggesting a link between auditory processing style and emotional intelligence.
What Problem Does This Paper Address?
Understanding how humans perceive and interpret emotional cues in speech is crucial for fields ranging from human-computer interaction to psychology. While we know that vocal prosody (intonation, rhythm, stress) carries significant emotional information, there's a question of why some individuals are better at decoding these cues than others. Previous research has identified two main auditory processing styles: "FO listeners" (or synthetic listeners) who focus on the fundamental frequency, and "spectral listeners" (or analytic listeners) who attend to the higher harmonics. These styles have even been linked to neurological differences in the brain's auditory cortex. However, the direct impact of these individual listening preferences on the ability to identify emotions in speech remained unexplored. This paper aims to bridge that gap, hypothesizing that since emotional prosody processing is generally right-hemisphere lateralized, spectral listeners might perform better on emotion identification tasks.
Methods
The study employed a two-part experimental design:
- Pitch Discrimination Task: This task was used to classify participants' preferred listening mode. Participants were presented with pairs of complex tones that either had a missing fundamental frequency or were constructed to create ambiguity. The perceived sequence of tones (rising or falling) could depend on whether the listener focused on the fundamental frequency (synthetic interpretation) or the spectral information (analytic interpretation). Participants' responses were used to calculate a "pitch perception preference index" ($d_p$), ranging from -1 (exclusively spectral) to +1 (exclusively FO). Crucially, the duration of these tones was manipulated (160 ms vs. 600 ms) to investigate its effect on listening preference.
- Emotion Judgment Task: Participants listened to short audio clips of sustained vowels (/a/) expressing eight different emotions (joy, anger, pride, panic fear, pleasure, irritation, anxiety, and relief). They had to identify the emotion using a forced-choice method. The primary outcome measure was the percentage of correctly identified emotions.
Datasets
The study utilized the Geneva Multimodal Emotion Portrayals (GEMEP) corpus for the emotion judgment task. This corpus contains emotional expressions from actors, carefully selected and rated for their emotional content. The researchers selected eight specific emotions from this corpus, including four basic emotions (anger, panic fear, joy, sadness) and four non-basic emotions (relief, pride, anxiety, pleasure).
Key Insights and Ideas
- Spectral Listeners Excel at Emotion ID: The most striking finding is the strong correlation between a spectral listening preference and better performance in identifying emotions in speech. This supports the hypothesis that how we process pitch information influences our ability to decode emotional cues.
- Duration Matters for Listening Mode: Longer stimulus durations (600 ms) significantly amplified the preference for a spectral listening mode compared to shorter durations (160 ms). This suggests that the temporal characteristics of auditory stimuli can influence our dominant processing strategy.
- Basic Emotions More Recognizable: The study found that basic emotions (anger, panic fear, joy, sadness) were generally better recognized than non-basic emotions when expressed in a sustained vowel. This might be due to the acoustic profiles of basic emotions being more robustly encoded in simpler vocalizations.
- Neural Basis Implied: The findings align with existing neurological evidence suggesting that spectral listeners have a rightward asymmetry in the "pitch processing center" of the brain, and that emotional prosody processing is also right-hemisphere lateralized.
Key Results
- A significant moderate negative correlation was found between the pitch perception preference index ($d_p$) and the proportion of correctly identified emotions. This means that as $d_p$ became more negative (indicating a stronger spectral preference), emotion identification accuracy increased.
- This correlation was stronger and more significant for longer stimulus durations (600 ms).
- Participants with a spectral listening mode performed better on the emotion judgment task, with a mean $d_p$ of -.69 for the seventy participants who met the quality criteria.
- Longer stimulus durations (600 ms) led to significantly better performance on control stimuli in the pitch discrimination task and a stronger spectral listening preference ($d_p$ = -0.80 vs. -0.59 for 160 ms).
- Female participants performed slightly better in emotion identification than male participants.
- Basic emotions were recognized with a significantly higher mean proportion correct (0.14 difference) than non-basic emotions.
Limitations and Future Work
- Limited Emotion Set: The study focused on only eight emotions, expressed as sustained vowels. It remains to be seen if the observed relationship between listening mode and emotion identification holds for a wider range of emotions or for more naturalistic, dynamic speech.
- Stimulus Simplification: Using sustained vowels simplifies the acoustic realization of emotions, potentially favoring emotions with distinct spectral or pitch characteristics (like anger or relief) over those that rely more heavily on dynamic prosodic changes (like pleasure).
- Participant Bias: The participant pool showed a strong bias towards spectral listeners, which might limit the generalizability of findings related to FO listeners. The authors note that instructions in the pitch discrimination task could influence this bias.
- Future Directions: The authors suggest replicating these findings with a broader set of emotions and more naturalistic speech stimuli. Further research could also explore the specific acoustic features that spectral listeners are more attuned to when decoding emotional prosody. Investigating the neural underpinnings of this relationship more directly would also be valuable.
Summary updated