TL;DR
This paper proposes a novel method for estimating a speaker's head orientation using only two microphones. Instead of relying solely on the peak of the cross-power spectrum phase (CSP) coefficients, the authors leverage the shape of the entire CSP vector, which captures reverberation effects influenced by head orientation. This allows for more robust estimation, especially in environments with echoes, and achieves high accuracy with minimal training data.
What Problem Does This Paper Address?
Understanding who is speaking and who they are talking to is crucial for advanced human-computer interaction, especially in multi-person scenarios like meetings or robotic communication. While speaker localization (figuring out where someone is) has been extensively studied, estimating a speaker's head orientation (figuring out where they are looking) is a more recent and challenging area. Existing methods often require complex microphone arrays spread throughout a room, which can be impractical. Some approaches use the peak value of CSP coefficients, but this is sensitive to direct sound and less effective in reverberant environments. This paper aims to address the need for a simpler, more robust head orientation estimation method that works with just two microphones and is less affected by room acoustics.
Methods
The core of the proposed method lies in using the shape of the Cross-Power Spectrum Phase (CSP) coefficients as a feature vector for discrimination. CSP analysis, often used for sound source localization, typically focuses on the peak value, which corresponds to the direct sound path. However, in real environments, reverberations (echoes) also contribute to the CSP. The authors hypothesize that these reverberation characteristics are influenced by both the speaker's position and their head orientation.
Their approach involves:
- Feature Extraction: Calculating CSP coefficients from the signals of two microphones. Instead of just the peak, they use a significant portion (up to 512 dimensions) of the CSP vector as the feature.
- Training: Recording speech data from various speaker positions and head orientations. For each utterance, the CSP coefficients are computed and used to train a Support Vector Machine (SVM) classifier. The SVM learns to associate specific CSP shapes with particular position-orientation pairs.
- Estimation: For a new, unknown utterance, the CSP coefficients are calculated, and the trained SVM is used to discriminate the most likely position and head orientation.
This method differs from previous work by explicitly utilizing the "shape" of the CSP coefficients, which implicitly encodes information about the acoustic transfer function and its dependence on head orientation and room acoustics.
Datasets
The experiments were conducted in a real room environment. The speech data used was from a male speaker from the ATR Japanese speech database. For training and testing, 50 words were recorded for each of the 48 possible position-orientation combinations (6 positions x 8 orientations). 10 of these words were used for testing, and the remaining 40 were used for training, with varying numbers of training data (1, 5, 10, 20, 30, 40 words) to evaluate performance.
Key Insights and Ideas
- Reverberation as a Feature: The key insight is that the subtle variations in reverberation, captured by the shape of the CSP coefficients, are highly indicative of head orientation. This moves beyond just the direct sound path.
- CSP Shape is Richer than Peak: Relying solely on the peak CSP value is insufficient for robust head orientation estimation, especially in reverberant conditions. The full CSP vector provides a more comprehensive acoustic fingerprint.
- Minimal Hardware Requirement: The method's effectiveness with just two microphones is a significant advantage over systems requiring extensive microphone arrays.
- Data Efficiency: The proposed method achieves high accuracy with a relatively small amount of training data (as few as 5 words per class).
Key Results
- High Accuracy: The method achieved a localization and head orientation estimation accuracy of over 90% with more than 5 training words, and close to 100% with more than 30 training words.
- Dimensionality Matters: Using more dimensions of the CSP coefficients (beyond just the peak) significantly improved head orientation estimation accuracy. Performance plateaued around 200-300 dimensions.
- Robustness to Noise: The method's performance degraded significantly in noisy environments (SNR of 5-10 dB), with accuracy dropping to chance levels. However, at a moderate SNR of 20 dB, accuracy was still substantial (47.5% mean).
- Confusion Matrix Insights: When only the peak value was used, orientations like 45° and 135° were often misclassified as 90°, suggesting the peak value alone couldn't distinguish these subtle differences.
Limitations and Future Work
- Noise Sensitivity: The most significant limitation identified is the drastic performance drop in low Signal-to-Noise Ratio (SNR) conditions. The authors note that the added pink noise caused CSP coefficients to have a high peak value at zero lag, leading to misclassifications.
- Future Work:
- Developing methods to discriminate between speech and noise to improve robustness.
- Investigating performance in different recording environments.
- Exploring estimation for unknown positions or head orientations (i.e., generalization beyond trained data).
Summary updated