Similarity of effects of emotions on the speech organ configuration with and without speaking

Tatsuya KitamuraInterspeech 2010

TL;DR

This paper proposes that emotional speech quality arises from unconscious physical changes in the speech organs, not deliberate articulation targets. They provide evidence using MRI and EMA that emotions alter speech organ configuration even when not speaking, and these changes are similar whether speaking or not, supporting their "side effect" hypothesis.

What Problem Does This Paper Address?

While it's widely accepted that emotions influence speech, the precise mechanism remains debated. Previous research has focused on how emotions affect the articulation of speech (i.e., how we consciously move our mouths and tongues to produce sounds). However, this paper tackles a less explored question: do emotions cause physical changes in the speech organs themselves, and do these changes influence voice quality independently of deliberate articulation, even when someone isn't speaking? The authors hypothesize that emotional arousal leads to unconscious physical alterations in the vocal tract, and these alterations are the primary drivers of emotional voice quality, acting as a "side effect" of the emotion rather than a consciously controlled articulatory goal.

Methods

The researchers employed two primary experimental methods:

  • Magnetic Resonance Imaging (MRI): This technique was used to capture detailed midsagittal images of the speech organs (vocal tract, tongue, jaw, larynx, pharynx). Participants, professional actors, were asked to simulate four emotions (neutral, hot anger, joy, sadness) in two conditions: while silent (nonspeaking) and while uttering the Japanese filler sound "/eh/" (speaking). This allowed for direct visualization of structural changes.
  • Electromagnetic Articulography (EMA): This method tracks the movement of small sensors attached to the articulators (lips, tongue, jaw). A participant watched either a relaxing or a horror movie to elicit spontaneous emotions. EMA data was collected during non-speech segments and during the utterance of "/eh/". This allowed for the measurement of articulatory changes under more naturalistic emotional states.

Datasets

Not discussed in the paper. The study utilized data collected from professional actors and one non-actor participant during controlled experiments.

Key Insights and Ideas

  • Emotions physically alter the vocal tract, even without speaking: The most striking finding is that emotional states can cause measurable changes in the shape and configuration of the speech organs (e.g., narrowing of the pharyngeal and laryngeal cavities, changes in jaw position) even when the participant is not producing speech sounds.
  • "Side Effect" Hypothesis: The paper strongly supports the idea that emotional voice quality is an unconscious consequence of these physical changes, rather than a deliberate attempt to sound emotional. This implies that speakers have little conscious control over the articulatory aspects of emotional speech.
  • Consistency across speaking states: A crucial observation is the similarity in speech organ deformation between the nonspeaking and speaking conditions for certain emotions. This suggests that the physical changes induced by emotion persist and influence articulation when speech is produced.
  • Individual differences in emotional expression: The study noted that while some general patterns emerged (e.g., laryngeal cavity narrowing in anger), the specific ways emotions affected speech organ configuration varied between individuals, highlighting the complexity of human emotional expression.

Key Results

  • MRI Experiment: For hot anger, the female participant showed a pulled-back lower jaw, narrowed pharyngeal/laryngeal cavities, and higher larynx. For joy, the jaw was pulled down, and cavities dilated. The male participant showed different patterns for anger (tongue dorsum up/forward, lower larynx) but a common narrowing of the laryngeal cavity with the female participant. Sadness showed blurring, indicating movement.
  • EMA Experiment: During the nonspeaking phase while watching a horror movie (Condition 2), the participant's tongue markers and lower jaw were displaced backward and downward compared to the relaxation movie (Condition 1), suggesting narrowed pharyngeal/laryngeal cavities. However, during the "/eh/" utterance, no significant differences in marker positions were found between the conditions. This latter finding suggests that by the time the participant responded verbally, they may have recovered from the immediate emotional impact.

Limitations and Future Work

  • Limited participant pool: The MRI experiment relied on professional actors, which may not generalize to the general population. The EMA experiment used only one participant.
  • Spontaneous emotion recovery: In the EMA experiment, the participant's emotional state may have diminished by the time they responded verbally, masking the effects of emotion on speaking articulation.
  • Future Work: The authors suggest the need for improved experimental designs to ensure participants maintain a specific emotion throughout the measurement period, especially during verbal responses. Further research could explore a wider range of emotions and participants, and investigate the precise acoustic correlates of these observed physical changes.

Summary updated

BibTeX
@inproceedings{kitamura2010similarity,
  author = {Tatsuya Kitamura},
  title = {{Similarity of effects of emotions on the speech organ configuration with and without speaking}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure