Distortion-controlled Training for End-to-end Reverberant Speech Separation with Auxiliary Autoencoding Loss

Yi Luo, Cong Han, Nima Mesgarani

Abstract

The performance of speech enhancement and separation systems in anechoic environments has been significantly advanced with the recent progress in end-to-end neural network architectures. However, the performance of such systems in reverberant environments is yet to be explored. A core problem in reverberant speech separation is about the training and evaluation metrics. Standard time-domain metrics may introduce unexpected distortions during training and fail to properly evaluate the separation performance due to the presence of the reverberations. In this paper, we first introduce the "equal-valued contour" problem in reverberant separation where multiple outputs can lead to the same performance measured by the common metrics. We then investigate how "better" outputs with lower target-specific distortions can be selected by auxiliary autoencoding training (A2T). A2T assumes that the separation is done by a linear operation on the mixture signal, and it adds an loss term on the autoencoding of the direct-path target signals to ensure that the distortion introduced on the direct-path signals is controlled during separation. Evaluations on separation signal quality and speech recognition accuracy show that A2T is able to control the distortion on the direct-path signals and improve the recognition accuracy.

Introduction

While speech separation in anechoic settings has reached a high level of maturity, reverberant environments remain a significant challenge. In a typical room, a microphone captures the direct-path signal followed by complex reflections. Standard training metrics like Signal-to-Noise Ratio (SNR) and Signal-to-Distortion Ratio (SI-SDR) struggle in these settings due to the "equal-valued contour" problem.

Geometric representation of the equal-valued contour problem in SNR space.

As illustrated in the geometry of SNR space, multiple estimated signals can lie on the same contour (a hyperball), resulting in identical scores despite varying levels of phase distortion or artifacts. End-to-end models lack an inherent mechanism to prefer a "cleaner" direct-path preservation over a distorted version if both satisfy the metric's mathematical criteria. This ambiguity often leads to outputs that, while technically accurate by SNR standards, perform poorly in downstream tasks like Automatic Speech Recognition (ASR).

Methodology: Auxiliary Autoencoding Training (A2T)

To mitigate this, we propose Auxiliary Autoencoding Training (A2T). This method leverages the fact that most state-of-the-art separation models, such as TasNet, utilize a linear mapping—typically by applying a mask to the input mixture.

The A2T framework introduces a dual-loss objective:

  1. Separation Loss: The standard objective forcing the model to extract the reverberant target from the mixture.
  2. Preservation Loss: A novel term that requires the model to perform "autoencoding" on the direct-path signal alone. This effectively mandates that if the model were provided with only the clean direct-path speech, its linear operator should act as an identity mapping, leaving the signal untouched.

To manage the training dynamics, where the simpler autoencoding task might dominate gradients, we employ Gradient Balancing techniques ($\alpha$-skewed SI-SDR and $\alpha$-thresholded SNR). This allows for precise tuning of the model's priority between the primary separation task and the auxiliary preservation task.

Experimental Setup

The proposed method was evaluated on a simulated two-speaker noisy reverberant dataset. Source speech was sampled from the LibriSpeech train-clean-100 subset, with noise from the 100 Nonspeech Corpus. Using the image method, we generated 28,000 unique room configurations. The direct-path signal was defined as the first ±6 ms relative to the peak of the room impulse response (RIR).

Results and Discussion

Testing was conducted using a DPRNN-TasNet architecture, evaluating both signal quality and Word Error Rate (WER) via a pre-trained ASR model. A2T demonstrated a significant improvement in Target-SNR (TSNR), which specifically measures the preservation of the direct-path signal. While the baseline model achieved a TSNR of ~12.5 dB, the A2T-enhanced model reached ~15.2 dB.

Word Error Rate (WER) performance across different overlap ratios (OR).

The reduction in distortion directly translated to improved ASR accuracy. As shown in the WER results, the baseline model's overall WER of 34.2% was reduced to 32.1% with A2T (at $\alpha=0.3$). Notably, the performance gains were most pronounced in high-overlap scenarios (OR $\in [75, 100]\%$), where standard metrics typically provide the least reliable gradients.

Conclusion

A2T effectively addresses the equal-valued contour problem by enforcing distortion control on the direct-path signal. While currently optimized for linear mapping architectures, the method significantly improves speech recognition accuracy in complex reverberant environments. Future research will focus on extending A2T to non-linear synthesis models and developing adaptive gradient balancing strategies based on real-time speaker overlap characteristics.

Summary updated

Abstract

The performance of speech enhancement and separation systems in anechoic environments has been significantly advanced with the recent progress in end-to-end neural network architectures. However, the performance of such systems in reverberant environments is yet to be explored. A core problem in reverberant speech separation is about the training and evaluation metrics. Standard time-domain metrics may introduce unexpected distortions during training and fail to properly evaluate the separation performance due to the presence of the reverberations. In this paper, we first introduce the "equal-valued contour" problem in reverberant separation where multiple outputs can lead to the same performance measured by the common metrics. We then investigate how "better" outputs with lower target-specific distortions can be selected by auxiliary autoencoding training (A2T). A2T assumes that the separation is done by a linear operation on the mixture signal, and it adds an loss term on the autoencoding of the direct-path target signals to ensure that the distortion introduced on the direct-path signals is controlled during separation. Evaluations on separation signal quality and speech recognition accuracy show that A2T is able to control the distortion on the direct-path signals and improve the recognition accuracy.
BibTeX
@inproceedings{luo2021distortion,
  author = {Yi Luo and Cong Han and Nima Mesgarani},
  title = {{Distortion-controlled Training for End-to-end Reverberant Speech Separation with Auxiliary Autoencoding Loss}},
  booktitle = {Slt},
  year = {2021},
  doi = {10.1109/slt48900.2021.9383494},
}
Figure