Session variability contrasts in the MARP corpus

Keith W. Godin, John H. L. HansenInterspeech 2010

TL;DR

This paper investigates the interplay between short-term (within-session) and long-term (between-session) speech variability in a multi-year corpus. They find that short-term variability, particularly conversational tone, plays a dominant role in distinguishing speech segments, even overshadowing longer-term changes that might be expected to define distinct sessions. This suggests that performance issues in speech systems attributed to session variability might be more influenced by subtle, ongoing changes in speech production rather than distinct session-defining events.

What Problem Does This Paper Address?

Speech recognition and speaker identification systems often struggle when encountering speech data from a different session than the one used for training. This "intersession variability" is a well-known challenge. While factors like microphone changes or channel noise are often blamed, this paper focuses on understanding the contribution of the speaker's own speech production behavior to this variability. Specifically, they aim to disentangle whether performance drops are due to changes within a session (intrasession variability) or changes between sessions (intersession variability), and how these relate to the temporal nature of the data. They propose that subtle, medium-term changes (like fatigue or mood) might be more influential than previously thought, and that these might be masked by more dominant short-term variations.

Methods

The paper employs two main experimental approaches using the MARP corpus:

  1. Clustering 30-second chunks:
    • Each session is divided into 30-second segments.
  2. Gaussian Mixture Models (GMMs) are trained for each segment.
  3. A hierarchical clustering approach is used, iteratively merging the two most similar chunks (based on symmetric Kullback-Leibler divergence) until only 20 clusters remain per speaker.
  4. The composition of these clusters (how many sessions they draw from) is analyzed to determine the dominance of intrasession vs. intersession variability.
  5. Session Verification:
    • This experiment adapts speaker verification to differentiate between sessions of the same speaker.
  6. 30-second chunks from sessions are used for testing.
  7. Models are trained using three different strategies: "first-half," "second-half," and "interleaved" subsets of sessions.
  8. The performance (measured by Equal Error Rate - EER) across these training regimes is compared to infer the impact of unmodeled intrasession variability.
  9. Maximum A Posteriori (MAP) adaptation from a speaker-specific Universal Background Model (UBM) is used for session modeling.

Datasets

The primary dataset used is the Multi-session Audio Research Project (MARP) corpus. This corpus contains speech recorded over a period of three years from multiple speakers, with sessions recorded at 3-6 month intervals. For this study, they utilize the conversational portions of 17-19 sessions from 14 speakers. No new benchmarks are introduced; the MARP corpus itself is the focus.

Key Insights and Ideas

  • Intrasession variability dominates: The clustering experiment reveals that 30-second chunks of speech are more likely to be related to other chunks from the same session than from different sessions, suggesting intrasession variability is a stronger factor in distinguishing short speech segments.
  • Conversational tone as a key intrasession factor: The "purity" analysis in the clustering experiment indicates that conversational tone (approximated by the proportion of chunks from the beginning, middle, or end of a session) is a measurable aspect of intrasession variability captured by their GMM modeling.
  • Aging is not the primary driver of intersession variability: Consistent with prior work, the session verification experiment suggests that the temporal separation between sessions is not a strong predictor of similarity. This implies that factors other than simple aging are responsible for intersession differences.
  • Unmodeled intrasession variability impacts session verification: While session verification is possible, the interleaved training regime (which better captures the temporal flow of a session) yields slightly better performance than training on only the first or last half. This small improvement suggests that unmodeled intrasession variability can indeed degrade performance, even if it's not the dominant factor.

Key Results

  • Clustering: The average number of sessions contributing to a cluster was 12.9, significantly higher than what would be expected from random clustering, indicating that intrasession relationships are strong. The empirical probability mass function of cluster composition differed significantly from a theoretical random model.
  • Session Verification: The interleaved training regime achieved a mean EER of 20.04% with a standard deviation of 6.30%, showing a slight improvement over the first-half (21.10% EER, 6.80% std. dev.) and second-half (21.28% EER, 6.69% std. dev.) training regimes. This indicates that while session identity can be captured, unmodeled intrasession variability plays a role.
  • Confusion Matrices: The confusion matrices for session verification showed that while sessions can be distinguished, there were also some consistent confusions, particularly between sessions 6 and 7, and the first 5 sessions showed less clear separation based on temporal distance.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the definitions of intrasession and intersession variability are working frameworks with only "vague association with phenomena of speech production."
    • The "conversational purity" measure is a rough approximation.
  • The MARP corpus, while controlled, may not fully represent real-world variability (e.g., microphone changes are excluded).
  • Future Work:
    • The authors suggest that future studies on intersession variability and speaker identification systems should consider grouping sessions based on similarities in medium-term factors like fatigue, mood, or circadian rhythm, rather than solely chronological order.
  • Further investigation into the specific mechanisms causing medium-term variability is warranted.

Summary updated

BibTeX
@inproceedings{godin2010session,
  author = {Keith W. Godin and John H. L. Hansen},
  title = {{Session variability contrasts in the MARP corpus}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure