Age, hearing loss and the perception of affective utterances in conversational speech

Juliane Schmidt, Esther Janse, Odette ScharenborgInterspeech 2014

TL;DR

This study investigates how age and hearing loss affect the perception of emotional nuances (arousal and valence) in natural conversational speech. The findings suggest that while age influences how individuals interpret affective cues, individual hearing loss among older adults did not significantly impact their perception in this study.

What Problem Does This Paper Address?

Understanding how humans perceive emotions in speech is crucial for developing more natural and empathetic human-computer interactions. While previous research has explored emotion perception, this paper tackles a specific gap: the combined influence of age and hearing loss on interpreting affective utterances in natural, non-acted conversational speech. Existing studies often use acted speech, which may not accurately reflect real-world emotional expression, and have not consistently accounted for individual hearing loss in older adults. This study aims to disentangle these factors by using a dimensional approach to emotion (arousal and valence) and analyzing natural speech, while also measuring participants' hearing thresholds.

Methods

The researchers employed a dimensional approach to emotion perception, focusing on two key axes: arousal (calm vs. aroused) and valence (positive vs. negative attitude). They used natural conversational speech fragments from the "Vera am Mittag" (VAM) corpus. Participants, divided into two age groups (younger and older adults), rated these speech fragments on a 5-point scale for either arousal or valence.

Key methodological aspects include:

  • Dimensional Rating: Instead of categorizing emotions (e.g., happy, sad), participants rated the degree of arousal and valence, allowing for a more nuanced understanding of affect.
  • Natural Speech Stimuli: The use of utterances from a German TV talk show (VAM corpus) aimed to capture the ecological validity of affective speech perception, contrasting with studies using acted speech.
  • Acoustic Analysis: The study correlated acoustic parameters (mean F0, mean intensity, articulation rate) with both the ground truth ratings of the VAM corpus and the participants' own ratings.
  • Hearing Assessment: Participants' hearing was assessed using pure-tone audiometry, and individual hearing loss was quantified to investigate its impact on affective perception in older adults.
  • Statistical Modeling: Linear mixed-effects regression analyses were used to model the relationships between acoustic parameters, age group, and affective ratings, while accounting for individual differences.

Datasets

The study utilized utterances from the "Vera am Mittag" (VAM) corpus. This corpus contains audio-only sections of conversational speech from a German TV talk show, providing affective utterances with pre-existing ground truth ratings for arousal and valence. The researchers selected subsets of utterances that were shorter than three seconds and semantically neutral to minimize interference from verbal content.

Key Insights and Ideas

  • Age Matters, Hearing Less So (in this sample): The study found significant age-related differences in how affective utterances are perceived. Older adults tended to rate utterances as less aroused and showed a weaker influence of intensity on their arousal ratings compared to younger adults. However, individual hearing loss among the older adults did not appear to be a significant factor in their affective perception ratings.
  • Acoustic Cues Differentially Predict Affect:
    • Arousal: Higher mean intensity was consistently associated with higher arousal ratings for both age groups. However, older adults showed a less pronounced effect of intensity on their arousal perception.
  • Valence: Higher mean F0 was associated with more negative valence ratings across both age groups. Interestingly, older adults seemed to interpret increasing intensity as more negative, a pattern not observed in younger adults.
  • Natural Speech is Key: The use of natural speech, with its subtler acoustic cues, highlights the challenges in affective perception, especially for older adults, compared to potentially overacted stimuli.

Key Results

  • Arousal:
    • Younger participants' arousal ratings were more strongly correlated with the ground truth than older participants'.
  • Higher mean intensity led to higher arousal ratings, but this effect was smaller for older adults.
  • Older adults rated utterances as less aroused overall compared to younger adults.
  • Valence:
    • Higher mean F0 was associated with more negative valence ratings for both age groups.
  • Older adults showed a significant interaction between age group and mean intensity, suggesting they interpreted increasing intensity as more negative.
  • Hearing loss did not significantly affect valence ratings or interact with acoustic parameters in the older adult group.

Limitations and Future Work

The authors acknowledge several limitations:

  • Sample Size: The study involved a relatively small sample size (10 participants per age group), which may limit the generalizability of findings, particularly regarding the impact of hearing loss.
  • Hearing Loss Range: The hearing loss among the older adult participants was not severe, and future research should include a broader range of hearing loss to better investigate its potential effects.
  • Stimulus Characteristics: While natural speech was used, the VAM corpus was noted to have an emphasis on neutral to more negative emotions due to its talk show origins.

Future work should aim to:

  • Include a larger and more diverse sample of participants with varying degrees of hearing loss.
  • Further explore the interaction between age, hearing, and the perception of subtle affective cues in natural speech.
  • Investigate other potential factors that might contribute to age-related differences in affect perception beyond hearing loss.

Summary updated

BibTeX
@inproceedings{schmidt2014age,
  author = {Juliane Schmidt and Esther Janse and Odette Scharenborg},
  title = {{Age, hearing loss and the perception of affective utterances in conversational speech}},
  booktitle = {Interspeech},
  year = {2014},
}
Figure