Abstract
Traditional Deep Neural Network (DNN) based speech separation methods typically rely on Minimum Mean Squared Error (MMSE) training, which assumes uniform variance across all frequency components of the prediction error. This paper introduces a Maximum Likelihood (ML) framework that models prediction errors as a multivariate Gaussian distribution with a learnable diagonal covariance matrix. This approach, termed ML-DNN, allows the model to adaptively weight frequency components based on their statistical reliability. Experimental results on the Speech Separation Challenge (SSC) corpus demonstrate that ML-DNN achieves faster convergence, superior reconstruction quality, and better generalization compared to standard MMSE-based models.
Introduction
Single-channel speech separation, often referred to as the "cocktail party problem," involves isolating a target speaker's voice from a single microphone recording containing overlapping speech. Recent advancements have utilized Deep Neural Networks (DNNs) to perform nonlinear mapping from the log-power spectral (LPS) features of mixed speech to those of the clean target speech.
However, the standard training objective for these networks is the Minimum Mean Squared Error (MMSE). MMSE implicitly assumes that the prediction error for every frequency component follows a Gaussian distribution with identical variance. In practice, different parts of the speech spectrum exhibit varying levels of predictability and noise. By treating all errors equally, MMSE models often fail to capture fine spectral details and may suffer from slow convergence and poor generalization to unseen speakers.
Methodology
To address the limitations of MMSE, we propose a Maximum Likelihood (ML) training objective. In this framework, the prediction error is modeled as a random variable following a multivariate Gaussian distribution $N(0, \Sigma)$.
The ML-DNN Architecture
The proposed architecture maps the LPS features of mixed speech to the target and interfering speakers simultaneously.
Key innovations include:
- Learnable Covariance ($\Sigma$): Unlike the fixed identity matrix assumed by MMSE, the ML-DNN learns a diagonal covariance matrix. This allows the model to automatically determine the "importance" or reliability of different frequency bands.
- Mahalanobis Distance Loss: The resulting negative log-likelihood loss function effectively becomes a weighted sum-of-squares, where weights are the inverse of the learned variances. This focuses the model's learning capacity on more predictable, low-variance components.
- Iterative Optimization: Training alternates between updating the DNN weights ($W$) via back-propagation and updating the covariance matrix ($\Sigma$) based on the current empirical prediction errors.
Experimental Setup
The model was evaluated using the Speech Separation Challenge (SSC) corpus. Mixed utterances were generated at Signal-to-Noise Ratios (SNRs) ranging from -10dB to 10dB. We extracted 257-dimensional LPS features for processing. Two primary evaluation modes were tested:
- Supervised: Both target and interfering speakers are known during training.
- Semi-supervised: The interfering speaker is unknown, testing the model's ability to generalize to generic interference.
Results and Discussion
The ML-DNN approach consistently outperformed the MMSE baseline across various objective metrics, including Output SNR, Short-Time Objective Intelligibility (STOI), and Perceptual Evaluation of Speech Quality (PESQ).
Convergence and Reconstruction
One of the most significant advantages of the ML approach is its efficiency in training. As shown in the reconstruction loss curves, the ML-DNN reaches a lower error floor significantly faster than the MMSE-DNN.
Performance Gains and Generalization
In supervised tests at -3dB input SNR, the ML-DNN achieved an output SNR of 9.62 dB, a substantial improvement over the 8.47 dB achieved by the MMSE model. Similar gains were observed in Source-to-Distortion Ratio (SDR) and Source-to-Artifacts Ratio (SAR), often exceeding 1.5 dB.
Crucially, the ML-DNN demonstrated superior generalization. Statistical analysis revealed a "variance gap" where prediction error variances are not equal across dimensions. By accounting for this, the ML-DNN consistently showed lower reconstruction loss on cross-validation sets compared to MMSE-DNN, even when the MMSE model had lower loss on the training set, proving the probabilistic approach prevents over-fitting to training noise.
Conclusion
By shifting from a rigid MMSE objective to a flexible Maximum Likelihood framework, we have demonstrated that DNNs can more effectively learn the complex statistical properties of speech separation. The ML-DNN not only converges faster but also provides superior generalization and reconstruction quality. Future research will explore more sophisticated error distributions to further enhance performance in highly non-stationary noise environments.
Summary updated