Abstract
This paper investigates the application of the Time-domain Audio Separation Network (TasNet), originally designed for multi-speaker separation, to the task of single-speaker speech enhancement (denoising). Through comparative analysis with standard uNet architectures, the authors demonstrate that TasNet achieves superior performance by learning an internal representation where speech and noise are more distinct than in the traditional Short-Time Fourier Transform (STFT) domain. However, the study also identifies a critical sensitivity to aliasing artifacts when using large frame hop sizes, highlighting a trade-off between computational efficiency and audio quality.
Introduction
Speech enhancement—the process of isolating a target voice from background noise—is a fundamental challenge for low-latency applications such as hearing aids and mobile communications. While TasNet has set new benchmarks in speech separation (extracting multiple voices), its efficacy for enhancement remained less explored. This study addresses three key questions: the performance gap between TasNet and established models like uNet, the underlying reasons for TasNet's success, and the impact of architectural parameters like hop size on performance.
Methods
The researchers compared two primary deep-learning architectures:
- TasNet: An end-to-end model comprising a linear Encoder, a Separator utilizing Temporal Convolutional Network (TCN) blocks, and a Decoder. It operates directly on time-domain waveforms.
- uNet: A standard encoder-decoder architecture featuring skip-connections, serving as a baseline for speech enhancement.
TasNet was evaluated in both non-causal (using Global Layer Normalization) and causal (using Cumulative Layer Normalization) configurations. Notably, the TasNet configuration employed a significantly larger receptive field (1.53s) compared to the uNet (0.32s).
Datasets
Evaluation utilized the Wall Street Journal (WSJ0) corpus for clean speech, combined with noise from CHiME3 and stationary speech-shaped noise (SSN). To ensure robust evaluation in multi-speaker scenarios, the authors introduced wsj0-2bal, a balanced version of the standard 2-speaker dataset that eliminates gender bias by providing an equal split of same-gender and opposite-gender pairs.
The "Inner-Domain" Advantage
A core finding of the study is that TasNet’s learned Encoder creates a representation where speech and noise overlap significantly less than in the STFT domain. Analysis showed that for a two-speaker mixture, signal overlap was reduced from 46% in STFT to just 20% in the TasNet domain. Similar reductions were observed for speech mixed with noise (SSN and BUS).
The Aliasing Trap
Despite its advantages, TasNet is highly sensitive to the "hop size" (the stride between audio frames). While the model excels with a 1ms hop size, performance degrades sharply at 32ms. The authors hypothesize that because the Separator processes rows of the inner domain independently, large hop sizes introduce aliasing artifacts—essentially decimation without filtering—that the model cannot rectify.
Key Results
TasNet consistently outperformed uNet across various noise conditions:
- Real-world Noise: In complex "mix" noise (bus, street, etc.), TasNet achieved an SI-SDR of 12.3 dB, surpassing uNet’s 11.1 dB.
- Competing Speakers: TasNet’s greatest advantage appeared when the "noise" was a competing speaker, reaching 16.0 dB SI-SDR compared to uNet's 13.4 dB.
- Causality: While moving to a causal (low-latency) system reduces performance, causal TasNet remains competitive with uNet.
Conclusion and Future Work
TasNet proves to be a powerful tool for single-speaker enhancement, largely due to its superior learned representations. However, the "aliasing bottleneck" remains a challenge for reducing computational load via larger hop sizes. Future research should explore Separator architectures that operate across rows of the inner domain to mitigate aliasing and further bridge the gap between causal and non-causal performance.
Summary updated