Perceptual Noise-Masking with Music through Deep Spectral Envelope Shaping

Clémentine Berger, Roland Badeau, Slim Essid

Abstract

This paper introduces a deep learning model that dynamically adjusts the equalization of music to mask ambient noise for headphone users. By leveraging psychoacoustic principles and a dual-constraint loss function, the system effectively hides background noise while minimizing unwanted changes to the music's original volume and artistic mix.

Introduction

Listening to music in noisy environments, such as public transport or busy streets, often results in a compromised experience as ambient noise masks the audio content. While Active Noise Cancellation (ANC) provides a partial solution, it cannot eliminate all residual noise. An alternative approach is to utilize the music itself to mask noise through the psychoacoustic phenomenon of simultaneous masking.

Previous methods for noise masking typically relied on simple volume increases—which risk hearing damage—or basic perceptual equalizers that often fail to account for the "spreading effect" (where sound in one frequency band masks adjacent bands). This paper proposes a smart, dynamic equalization system that shapes the music's spectral envelope to optimally hide noise without compromising the track's original loudness or fidelity.

Methodology

System Architecture

The proposed system, DPNMM (Deep Perceptual Noise Masking with Music), utilizes a U-Net-based neural network architecture. The model processes the Power Spectral Densities (PSDs) of both the music and the ambient noise, mapped to the Bark scale to mimic human auditory perception.

The DPNMM system pipeline integrates psychoacoustic features and a U-Net architecture to generate gain values that are then spread across the spectrum.

As shown in the pipeline above, the U-Net outputs gain values for each Bark band. These gains are converted into filter responses using a cosine-shaped pattern, which explicitly models the spreading effect. This approach allows the network to rely on the cumulative masking power of adjacent bands, reducing the need for aggressive amplification in any single band.

Dynamic Dual-Constraint Loss

To balance noise masking ($L_0$) with the preservation of original music power ($L_{power}$), the authors implemented a dynamic multiplier ($\lambda_t$). This multiplier adjusts based on whether the music's power variation exceeds a predefined threshold ($\Delta P_{max}$). This automated weighting allows the model to prioritize masking when within power limits and penalize deviations when the volume changes too drastically.

Experimental Results

The model was trained on 50 hours of simulated headphone-listening data using the DNS Challenge (noise) and FMA (music) datasets. Performance was evaluated using two primary metrics: Noise-to-Mask Ratio (NMR), which measures masking effectiveness, and Global Level Difference (GLD), which measures volume alteration.

Comparison of NMR (top) and GLD (bottom) metrics for DPNMM under various power constraints compared to the Estreder baseline across different noise types.

As illustrated in the results, DPNMM models with power constraints of $\Delta P_{max} = 1$ and $2$ dBA significantly outperformed the baseline perceptual graphic equalizer. These models achieved superior noise masking (lower NMR) while maintaining better fidelity to the original audio levels (lower GLD). The data demonstrates a clear trade-off: while unconstrained models achieve the best masking, they cause heavy volume distortion. Conversely, a constraint of $0.5$ dBA is often too strict to effectively mask low-frequency noise.

Conclusion

DPNMM demonstrates that deep spectral envelope shaping can effectively mask ambient noise while preserving the artistic integrity of music. While the model currently struggles with high-power low-frequency noise under strict constraints, it provides a robust framework for perceptual EQ. Future work will focus on incorporating individual user preferences to further personalize the listening experience.

Summary updated

BibTeX
@inproceedings{berger2025perceptual,
  author = {Clémentine Berger and Roland Badeau and Slim Essid},
  title = {{Perceptual Noise-Masking with Music through Deep Spectral Envelope Shaping}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10888409},
}
Figure