A microphone-independent visualization technique for speech disorders

Andreas Maier, Stefan Wenhardt, Tino Haderlein, Maria Schuster, Elmar NöthInterspeech 2009

TL;DR

This paper introduces a novel method to visualize speech disorders that is robust to variations in recording equipment. By extending the Sammon mapping technique, they can create consistent visualizations of speakers regardless of whether they were recorded with different microphones, which is crucial for large-scale, multi-site studies.

What Problem Does This Paper Address?

Visualizing speech data can be incredibly useful for understanding speech disorders. Techniques like Sammon mapping can project complex speech features into a 2D or 3D space, revealing relationships between speakers. However, a major challenge arises when speech data is collected under different recording conditions, particularly with different microphones. These variations can cause the same speaker to appear in vastly different locations on the map, obscuring the underlying speech characteristics and making it difficult to interpret the visualization. This is especially problematic for multi-site studies where consistent recording setups are not always feasible. The authors aim to overcome this "microphone effect" to create reliable and interpretable visualizations of speech disorders.

Methods

The core of the paper's contribution lies in an extension of the Sammon mapping algorithm. Standard Sammon mapping aims to preserve the distances between data points in a high-dimensional space when projecting them to a lower-dimensional space. The authors introduce a new term to the Sammon mapping's objective function that specifically penalizes distances between representations of the same speaker recorded under different conditions.

Here's a breakdown of the key components:

  • Feature Extraction: Speech is represented by parameters from speaker-adapted Gaussian mixture densities of a speech recognizer. These are essentially statistical models of the speech characteristics for each speaker.
  • Distance Metric: A Mahalanobis distance is used to quantify the difference between these Gaussian mixture models.
  • Standard Sammon Mapping: This technique reduces the dimensionality of the features to a 2D or 3D space while trying to preserve the calculated distances.
  • Extended Sammon Mapping (The Novelty): The authors modify the Sammon mapping's objective function by adding a "grouping error" term. This term explicitly encourages points representing the same speaker (even if recorded with different microphones) to be close together in the visualization. A weighting factor, Q, controls the trade-off between preserving the original distances (Sammon error) and minimizing the grouping error.

Datasets

The paper uses data from:

  • 31 children with cleft lip and palate (CLP): Recorded using a dnt Call 4U Comfort headset.
  • 87 children (control group): Recorded using a Plantronics Audio USB 510 headset.

Crucially, to create matched recordings for their microphone-independent approach, they replayed the speech data using a reference loudspeaker and recorded it again with the other microphone. This allowed them to generate datasets where the same speech was captured by both microphones.

Key Insights and Ideas

  • Microphones Dominate Visualizations: The paper clearly demonstrates that differences in microphones can create stronger clustering effects than the actual speech disorders being studied. This highlights a fundamental challenge in speech visualization.
  • The "Grouping Error" is Key: By explicitly penalizing the distance between representations of the same speaker recorded with different microphones, the extended Sammon mapping forces these points to converge. This is the core mechanism for achieving microphone independence.
  • Balancing Act with Q: The weighting factor Q is essential. A Q of 0 represents standard Sammon mapping, while a high Q prioritizes minimizing the grouping error. The authors found that a Q of 0.9 offered a good balance, significantly reducing grouping errors while maintaining meaningful correlations with speech disorder criteria.
  • Restoring Meaningful Correlations: The most compelling insight is that the extended Sammon mapping, with a high Q, successfully restores the correlation between the visualization's coordinates and perceptually rated criteria of speech disorders (like intelligibility), which was lost when using standard Sammon mapping with data from multiple microphones.

Key Results

  • Visual Separation of Microphones: Figure 2 and Figure 3 (a) clearly show that standard Sammon mapping results in distinct clusters for each microphone, even when the same speakers are involved.
  • Successful Grouping: With the extended Sammon mapping (e.g., Q=0.9), the points representing the same speaker are pulled together, and the distinct microphone clusters disappear (Figure 3(c)).
  • Correlation Restoration: Table 2 shows that while standard Sammon mapping (Q=0) with multi-microphone data shows some correlations with speech disorder criteria, these correlations drop significantly. The extended Sammon mapping (Q=0.9) successfully restores these correlations, particularly for "weakened plosives" and "intelligibility." For intelligibility, the correlation increases from 0.12 to 0.55 (**) with the extended method.
  • Trade-off between Errors: Figure 4 illustrates the trade-off: as the grouping error decreases with increasing Q, the Sammon error (representing the loss of information during dimensionality reduction) increases.

Limitations and Future Work

  • Normalization: The paper notes that the normalization in the grouping error calculation (Eq. 7) is performed with 1/N due to the sparsity of the group indicator matrix G. This might imply a simplification that could be further investigated.
  • Optimal Q: While Q=0.9 is presented as a good compromise, the paper suggests that the optimal Q might depend on the specific dataset and the desired balance between grouping and Sammon error.
  • Generalizability: The method is demonstrated on speech disorders. Future work could explore its applicability to other types of acoustic data where recording variations are a concern.
  • Online Demo: The authors mention an online demo is available, suggesting a desire for wider adoption and testing of their visualization technique.

Summary updated

BibTeX
@inproceedings{maier2009microphone,
  author = {Andreas Maier and Stefan Wenhardt and Tino Haderlein and Maria Schuster and Elmar Nöth},
  title = {{A microphone-independent visualization technique for speech disorders}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure