Design of Novel Field Programmable Gate Array-Based Hearing Aid

Bor-Shing Lin, Po-Yu Yang, Ching-Feng Liu, Yi-Chia Huang, Chengyu Liu, Bor-Shyh LinIEEE Access 2019 · DOI

Introduction

The "cocktail party problem"—the challenge of isolating a single voice in a noisy, crowded environment—remains one of the most significant hurdles in hearing aid technology. While traditional devices attempt to filter noise based on frequency, they often struggle when background chatter overlaps with the target speech. This paper, published in IEEE Access, proposes a hybrid solution: a hearing aid system that combines computer vision with adaptive audio filtering on a Field Programmable Gate Array (FPGA). By "seeing" the target speaker, the system can more accurately "tune in" to their voice.

The Localization Challenge

Standard hearing aids often use "adaptive beamforming," which employs multiple microphones to focus on a specific direction. However, this technique relies on calculating the Time Difference of Arrival (TDOA) of sound across the microphones. In real-world settings, reverberation (echoes) can confuse these acoustic algorithms, leading to poor tracking. The researchers addressed this by offloading the localization task to a camera, providing a precise and stable TDOA even in echoey rooms.

System Architecture and Methodology

The proposed system integrates three core components to achieve real-time speech enhancement.

1. Visual Localization

A wide-angle webcam captures the environment, and OpenCV’s Haar cascade classifiers are used to detect faces and eyes. The user manually selects the target speaker on a host interface. The system then estimates the speaker's exact distance and angle using linear regression models based on the pixel width of the speaker's eyes ($W_{eyes}$) and their horizontal offset ($d_{hor}$).

2. Adaptive Beamforming

The audio processing utilizes a four-microphone array and a Generalized Sidelobe Canceller (GSC) architecture. The visual TDOA data guides a "fixed" beam toward the speaker, while a "blocking matrix" identifies ambient noise. An Adaptive Signal Enhancement (ASE) block, utilizing a Normalized Least Mean Square (NLMS) algorithm, then subtracts the identified noise from the primary speech signal.

System setup: (a) conceptual diagram showing the host system, webcam, and microphone array targeting a desired speaker; (b) the actual experimental environment with three potential speakers.

3. FPGA Implementation

To meet the low-latency requirements of hearing assistance, the ASE algorithm was implemented on a Xilinx Artix-7 FPGA. The design employs fixed-point arithmetic and a four-state finite-state machine (FSM) to process audio at a 20 kHz sampling rate. This hardware implementation is highly efficient, utilizing only 27% of the FPGA's Look-Up Tables (LUTs) and 19% of its Flip-Flops.

Key Results

The system's performance was evaluated through both objective metrics and subjective participant testing:

  • SNR Improvement: Across various noisy environments, the system achieved an average Signal-to-Noise Ratio (SNR) improvement of approximately 5 dB. In the most challenging test conditions, the SNR was raised from -7.26 dB to -1.42 dB.
  • Speech Quality and Intelligibility: Perceptual Evaluation of Speech Quality (PESQ) scores improved from "Poor" (~1.5) to "Fair/Good" (~1.9). Subjective tests revealed that participant speech recognition thresholds improved from 88% to 96% with the enhancement active.
  • Precision: The visual TDOA estimation proved remarkably robust, with errors significantly lower than traditional acoustic-only methods in reverberant environments.

Conclusion and Future Directions

By leveraging visual data to solve the acoustic localization problem, this research provides a robust path forward for hearing aid technology. While the current prototype requires a host laptop, the efficiency of the FPGA implementation suggests that the system could eventually be integrated into wearable hardware, such as smart glasses. Future work will focus on automating speaker selection through eye-tracking and improving performance in low-light conditions.

Summary updated

BibTeX
@article{lin2019design,
  author = {Bor-Shing Lin and Po-Yu Yang and Ching-Feng Liu and Yi-Chia Huang and Chengyu Liu and Bor-Shyh Lin},
  title = {{Design of Novel Field Programmable Gate Array-Based Hearing Aid}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2916723},
}
Figure