Abstract
The Deep Noise Suppression (DNS) challenge is designed to foster innovation in the area of noise suppression to achieve superior perceptual speech quality. This is the 4th DNS challenge, with the previous editions held at INTERSPEECH 2020, ICASSP 2021, and INTERSPEECH 2021. We open-source datasets and test sets for researchers to train their deep noise suppression models, as well as a subjective evaluation framework based on ITU-T P.835 to rate and rank-order the challenge entries. We provide access to DNSMOS P.835 and word accuracy (WAcc) APIs to challenge participants to help with iterative model improvements. In this challenge, we introduced the following changes: (i) Included mobile device scenarios in the blind test set; (ii) Included a personalized noise suppression track with baseline; (iii) Added WAcc as an objective metric; (iv) Included DNSMOS P.835; (v) Made the training datasets and test sets fullband (48 kHz). We use an average of WAcc and subjective scores P.835 SIG, BAK, and OVRL to get the final score for ranking the DNS models. We believe that as a research community, we still have a long way to go in achieving excellent speech quality in challenging noisy real-world scenarios.
Introduction
In the era of hybrid work, background noise—from barking dogs to kitchen clatter—has become a major source of "meeting fatigue." While deep learning-based noise suppression has made massive strides, three major gaps remained: the "other voice" problem where neighboring talkers are amplified, the intelligibility trade-off where aggressive denoising makes speech sound robotic, and the need for higher fidelity 48 kHz fullband audio. The 4th DNS Challenge addresses these by moving beyond simple noise reduction to focus on clarity and personalization.
Methodology and Datasets
The challenge was split into two tracks. Track 1 focused on traditional non-personalized DNS for fullband audio. Track 2 introduced Personalized DNS (PDNS), which suppresses both background noise and interfering talkers. To achieve personalization, participants used 2.5 minutes of "enrollment speech" to extract a speaker embedding, allowing models to "lock onto" the primary voice.
The challenge provided a massive open-source repository including clean speech from over 3,200 talkers across six languages and a diverse noise dataset.
The noise dataset comprised approximately 62,000 clips across 150 classes, ensuring models were robust to common real-world sounds like typing, mouse clicks, and running water. Evaluation utilized the ITU-T P.835 subjective framework (SIG, BAK, OVRL) and Word Accuracy (WAcc) to ensure denoising didn't compromise machine intelligibility.
Results and Discussion
A key innovation in this challenge was the use of DNSMOS P.835, a neural network-based objective metric that mimics human ratings. The reliability of this metric was confirmed by its high correlation with human subjective scores.
As shown in the table above, the high Pearson Correlation Coefficient (PCC) values (0.92–0.96) validate DNSMOS as a powerful tool for rapid prototyping. Other key findings include:
- Personalization Advantage: Speaker-aware models in Track 2 significantly outperformed general models in crowded environments.
- Mobile Performance: MOS scores for clips recorded on mobile devices were higher than those from desktops, suggesting modern smartphone hardware provides a superior starting point for software-based DNS.
- The Intelligibility Balance: By incorporating WAcc into the final ranking, the challenge forced a balance between silence and clarity, discouraging models that "mangled" speech to achieve higher noise suppression.
- Efficiency: The winning model for Track 1 demonstrated that high performance is possible with a small footprint, using only 1.5M parameters.
Conclusion
While models significantly improved background noise (BAK), none of the submissions actually improved raw Speech Quality (SIG) or Word Accuracy (WAcc) compared to the original noisy clips. This highlights that the community still has a long way to go in achieving true high-fidelity speech in challenging environments. Future work will focus on increasing linguistic diversity and implementing stricter benchmarking for real-time latency and algorithmic delay.
Summary updated