Abstract
In this paper, we present a new approach for speaker recognition, which uses the prosodic information calculated on the original speech to resynthesize the new speech data utilizing the spectrum modeling technique. The resynthesized data are modeled with sinusoids based on pitch, vibration amplitude and phase bias. We use the resynthesized speech data to extract cepstral features for speaker modeling and scoring in the same way as in traditional speaker recognition approaches. We then model these features using GMMs and compensate for speaker and channel variability effects using joint factor analysis. The experiments are carried out on the core condition of NIST 2008 speaker recognition evaluation data. The experimental results show that our proposed system achieves comparable performance to the state-of-the-art cepstral-based joint factor analysis system which uses the original data for speaker recognition.
Introduction
Traditional speaker recognition systems primarily rely on acoustic features like Mel Frequency Cepstral Coefficients (MFCCs) to capture the unique spectral characteristics of a speaker's voice. While highly accurate, these features are susceptible to channel distortions and environmental noise. Conversely, prosodic features—such as pitch, energy, and rhythm—are more robust to such variations but often lack the discriminative power for high-performance recognition on their own. This research explores a novel method to integrate prosodic information by using it to resynthesize speech, creating a "purified" signal from which standard acoustic features can be more reliably extracted.
Methodology
The core of the proposed approach is a "Resynthesis-then-Recognition" pipeline. The process begins with sinusoidal modeling, where the input waveform is decomposed into its deterministic components.
As shown in the flowchart, the system performs pitch detection on the input waveform. If a valid pitch value is found, the harmonic structure is extracted. The speech is then resynthesized using these sinusoids, effectively filtering out non-harmonic noise and unvoiced segments that are prone to corruption. Standard MFCCs are then extracted from this resynthesized signal. To mitigate speaker and channel variability, Joint Factor Analysis (JFA) is applied to the Gaussian Mixture Model (GMM) backends.
Experimental Setup
The system was evaluated using the NIST 2008 Speaker Recognition Evaluation (SRE) core condition (short2-short3). Training data included a diverse set of corpora: NIST SRE 2004–2006, Switchboard II, and Switchboard Cellular. The evaluation focused on three primary scenarios:
- Telephone-to-Telephone (tel-tel)
- Interview-to-Interview (intmic-intmic)
- Interview-to-Telephone (intmic-tel), representing a challenging cross-channel condition.
Results and Discussion
The experimental results demonstrate that the fusion of systems trained on original and resynthesized data consistently outperforms the baseline original-only system.
The DET curves illustrate the performance gains across various conditions. Notably, in the difficult cross-channel (intmic-tel) task, the fusion system achieved a 24.7% improvement in the Minimum Decision Cost Function (minDCF). In the interview-to-interview (intmic-intmic) scenario, the improvement was even more pronounced at 38.7%. Even in standard telephone-to-telephone conditions, a gain of 8.4% in minDCF was observed.
These results suggest that the resynthesized speech captures a "purified" version of the speaker's voice. While some information from unvoiced segments is lost, the resulting features provide a highly complementary view of the speaker's identity, particularly in low Signal-to-Noise Ratio (SNR) environments.
Conclusion
This study demonstrates that prosodic information can be effectively leveraged for speaker recognition through speech resynthesis. By rebuilding the speech signal from its harmonic components, we create a robust foundation for acoustic feature extraction. While the current model discards inharmonic and unvoiced components, its ability to significantly enhance performance through system fusion highlights its value as a supplementary technique. Future research will focus on integrating stochastic components of speech to further refine the resynthesis process and recover potentially lost speaker-specific information.
Summary updated