Hearing-Loss Compensation Using Deep Neural Networks: A Framework and Results From a Listening Test

Peter Leer, Jesper Jensen, Laurel H. Carney, Zheng‐Hua Tan, Jan Østergaard, Lars Bramsløw

TL;DR

Researchers have developed a deep learning framework that personalizes hearing-loss compensation by "matching" the neural response of an impaired ear to that of a healthy one. By using neural networks to emulate complex biological auditory models, the team created a processing strategy that outperformed conventional hearing aid methods in both speech intelligibility and user preference during real-world listening tests.

What Problem Does This Paper Address?

Hearing loss is more than just a "volume" problem; it involves complex, non-linear distortions in how the auditory nerve transmits signals to the brain. Conventional hearing aids typically rely on "prescriptive formulas" (like NAL-R) that apply linear gain and compression based on an individual's audiogram. While helpful, these methods often fail to restore the full richness of sound or provide adequate clarity in noisy environments.

The challenge in using more sophisticated, biologically-inspired models is that they are computationally "expensive"—running a full simulation of the auditory nerve takes far too long for real-time hearing aid applications or even for training modern machine learning models. This paper tackles the gap between complex biological theory and practical, real-time signal processing.

Methods

The authors propose a multi-stage deep learning approach:

  1. Auditory Model Emulators (AMEs): Since biological models (like the Zilany model) are slow, the researchers trained a Wave-U-Net (a convolutional auto-encoder) to act as a high-speed proxy. These AMEs can predict the auditory nerve's firing rate in response to sound in milliseconds rather than minutes.
  2. DNN-based Hearing-Loss Compensation (DNN-HLC): They then trained a second neural network to "fix" the audio. This network takes an input signal and modifies it so that when it passes through a hearing-impaired AME, the resulting neural output matches the output of a normal-hearing AME. Essentially, it pre-distorts the sound to compensate for the ear's specific dysfunction.
  3. Linearization and Ripple Reduction: The authors discovered that discrete frequency channels in auditory models can cause a "comb-filter" or "ripple" effect in the audio gain, leading to artifacts. They developed a novel algorithm (Algorithm 1) to choose acoustic center frequencies that minimize these artifacts.
  4. Joint Processing: They also created a version (DNN-HLCNR) that performs both hearing-loss compensation and noise reduction simultaneously.

Datasets

  • Training: The models were trained using the LibriTTS dataset, consisting of 4,000 English speech utterances.
  • Noise: To ensure robustness, the researchers added 16-speaker babble noise and stationary Gaussian speech-shaped noise at various signal-to-noise ratios (SNRs).
  • Evaluation: The system was tested on 13 hearing-impaired human listeners using the Danish Hearing in Noise Test (HINT) for intelligibility and a MUSA (Multiple Stimuli with Anchor) test for subjective preference.

Key Insights and Ideas

  • Hearing Loss as an Optimization Problem: Instead of just turning up the volume, this approach treats hearing-loss compensation as a "neural matching" task. If we know how a healthy nerve responds to a sound, we can optimize the input signal to force an impaired nerve to produce that same response.
  • The "Ripple" Discovery: One of the most interesting technical findings was that the spacing of frequency channels in the auditory model directly impacts the smoothness of the resulting audio. By moving away from standard logarithmic spacing and using a "decay-based" spacing, they significantly reduced audible artifacts.
  • Cross-Language Generalization: Interestingly, the models were trained on English speech but successfully improved intelligibility for Danish listeners. This suggests the DNNs learned fundamental auditory compensation principles rather than just language-specific features.

Key Results

  • Intelligibility: In the HINT tests, the DNN-HLC provided a significant boost in speech clarity. It resulted in a 13 RAU (Rationalized Arcsine Unit) increase in intelligibility compared to unprocessed audio and outperformed the conventional NAL-R baseline.
  • Preference: In subjective testing, the joint compensation and noise reduction model (DNN-HLCNR) was the clear winner, achieving a median preference score of 94 out of 100, compared to just 50 for the conventional NAL-R approach.
  • Efficiency: By using AMEs, the researchers reduced the time needed to calculate the "gradient" of the auditory model by several orders of magnitude, making the training of these compensation networks feasible for the first time.

Limitations and Future Work

  • Fixed Sound Levels: The current study focused on a fixed input level of 65 dB SPL (normal conversation). Real-world hearing aids must handle a massive dynamic range, from whispers to shouting, which remains a challenge for this framework.
  • The "Black Box" of Loss Functions: While the model improved results, the authors note that we still don't fully understand which specific neural features (timing vs. rate) are most important for the brain to "decode" speech.
  • Real-time Constraints: While the inference is fast, the study did not focus on the strict latency requirements (usually <10ms) needed for wearable hearing aids. Future work will need to optimize these architectures for low-power, low-latency hardware.

Summary updated

Abstract

This article investigates the use of deep neural networks (DNNs) for hearing-loss compensation. Hearing loss is a prevalent issue affecting millions of people worldwide, and conventional hearing aids have limitations in providing satisfactory compensation. DNNs have shown remarkable performance in various auditory tasks, including speech recognition, speaker identification, and music classification. In this study, we propose a DNN-based approach for hearing-loss compensation, which is trained on the outputs of hearing-impaired and normal-hearing DNN-based auditory models in response to speech signals. First, we introduce a framework for emulating auditory models using DNNs, focusing on an auditory-nerve model in the auditory pathway. We propose a linearization of the DNN-based approach, which we use to analyze the DNN-based hearing-loss compensation. Additionally we develop a simple approach to choose the acoustic center frequencies of the auditory model used for the compensation strategy. Finally, we evaluate, to our knowledge for the first time, the DNN-based hearing-loss compensation strategies using listening tests with hearing impaired listeners. The results demonstrate that the proposed approach results in feasible hearing-loss compensation strategies. Our proposed approach was shown to provide an increase in speech intelligibility versus an unprocessed baseline and was found to outperform a conventional approach in terms of both intelligibility and preference.
BibTeX
@article{leer2025hearing,
  author = {Peter Leer and Jesper Jensen and Laurel H. Carney and Zheng‐Hua Tan and Jan Østergaard and Lars Bramsløw},
  title = {{Hearing-Loss Compensation Using Deep Neural Networks: A Framework and Results From a Listening Test}},
  journal = {IEEE Transactions on Audio Speech and Language Processing},
  year = {2025},
  doi = {10.1109/taslpro.2025.3536183},
}
Figure