Near-field signal acquisition for smartglasses using two acoustic vector-sensors

Dovid Y. Levin, Emanuël Anco Peter Habets, Sharon Gannot

Abstract

Smartglasses, in addition to their visual-output capabilities, often contain acoustic sensors for receiving the user's voice. However, operation in noisy environments may lead to significant degradation of the received signal. To address this issue, we propose employing an acoustic sensor array which is mounted on the eyeglasses frames. The signals from the array are processed by an algorithm with the purpose of acquiring the user's desired near-field speech signal while suppressing noise signals originating from the environment. The array is comprised of two Acoustic Vector Sensors (AVSs) which are located at the fore of the glasses' temples. Each AVS consists of four collocated subsensors: one pressure sensor and three particle-velocity sensors oriented in mutually orthogonal directions. The array configuration is designed to boost the input power of the desired signal and ensure diverse noise characteristics across channels. We propose an algorithm that incorporates time-frequency domain speech detection to adaptively update estimates of noise statistics. Experimental results show that the proposed algorithm performs favorably compared to conventional methods.

Introduction

Wearable computers, particularly smartglasses, increasingly rely on voice commands and hands-free telephony. However, capturing clean speech in noisy environments is challenging due to the extreme spatial constraints of the frame. Standard omnidirectional microphones placed close together capture highly correlated background noise, making spatial filtering difficult.

Furthermore, smartglasses introduce a unique dynamic challenge: while the distance from the wearer's mouth to the sensors remains constant, the user's head movement causes the spatial profile of background noise to change abruptly. This confuses standard static noise-canceling algorithms. This paper addresses these issues by leveraging the unique physics of Acoustic Vector Sensors (AVSs) and a specialized adaptive beamforming pipeline.

Proposed Methodology

To overcome spatial constraints, the system utilizes two AVSs placed at the front of the left and right temples. Unlike standard scalar pressure microphones, an AVS measures both pressure and particle velocity. Each sensor package provides four channels (one omnidirectional and three orthogonal dipoles), resulting in an 8-channel array from a very compact footprint.

Adaptive Beamforming Pipeline

The signal processing is based on a Minimum Variance Distortionless Response (MVDR) beamformer operating in the time-frequency domain.

Block diagram of the proposed adaptive beamforming algorithm, highlighting the noise covariance estimation and speech detection stages.

Key components of the pipeline include:

  1. Fixed Target Steering: Since the glasses do not move relative to the wearer's mouth, a fixed Relative Transfer Function (RTF) is used to steer the beamformer.
  2. Dynamic Noise Tracking: To account for head rotation, the noise covariance matrix is continuously updated.
  3. Narrowband Speech Detection: To prevent the wearer's voice from corrupting noise estimates, a narrowband test statistic determines speech presence in specific time-frequency bins. Noise updates are paused only in bins where speech is detected, allowing for more granular adaptation.
  4. Near-Field Advantage: The dipole sensors naturally amplify near-field sounds (like the wearer's voice) more than far-field noise due to frequency-dependent acoustic impedance, providing a physical SNR boost.

Experimental Evaluation

The system was evaluated in a controlled acoustic room using a Head and Torso Simulator (HATS) named "Cnut." Microflown USP probes served as the AVSs, mounted directly onto the glasses frames.

The experimental setup featuring the Head and Torso Simulator (HATS) equipped with the AVS-integrated smartglasses.

Testing involved five independent audio sources: the HATS internal speaker (target), three static external loudspeakers, and one moving human speaker. These were mixed to create scenarios with Signal-to-Noise Ratios (SNRs) ranging from -20 dB to +10 dB.

Results and Discussion

The inclusion of AVS hardware provided a significant performance leap. Utilizing all 8 channels of the two AVSs yielded a 15 to 25 dB improvement in noise reduction compared to using only the two standard pressure channels.

In dynamic scenarios with a moving noise source at a -20 dB SNR, the proposed adaptive algorithm achieved 29.3 dB of noise reduction, whereas a baseline fixed-MVDR approach only managed 16.1 dB. Crucially, the system maintained high Short-Time Objective Intelligibility (STOI) scores, indicating that the aggressive noise suppression did not significantly distort the target speech.

Conclusion

This research demonstrates that AVS-equipped smartglasses can achieve superior speech acquisition in noisy, dynamic environments. By exploiting the fixed geometry of the wearer's mouth and the unique near-field properties of vector sensors, the system provides substantial noise rejection without the need for complex source tracking. Future work will focus on refining post-processing filters and simplifying the initial calibration phase required to estimate the wearer's specific RTF.

Summary updated

Abstract

Smartglasses, in addition to their visual-output capabilities, often contain acoustic sensors for receiving the user's voice. However, operation in noisy environments may lead to significant degradation of the received signal. To address this issue, we propose employing an acoustic sensor array which is mounted on the eyeglasses frames. The signals from the array are processed by an algorithm with the purpose of acquiring the user's desired near-filed speech signal while suppressing noise signals originating from the environment. The array is comprised of two AVSs which are located at the fore of the glasses' temples. Each AVS consists of four collocated subsensors: one pressure sensor (with an omnidirectional response) and three particle-velocity sensors (with dipole responses) oriented in mutually orthogonal directions. The array configuration is designed to boost the input power of the desired signal, and to ensure that the characteristics of the noise at the different channels are sufficiently diverse (lending towards more effective noise suppression). Since changes in the array's position correspond to the desired speaker's movement, the relative source-receiver position remains unchanged; hence, the need to track fluctuations of the steering vector is avoided. Conversely, the spatial statistics of the noise are subject to rapid and abrupt changes due to sudden movement and rotation of the user's head. Consequently, the algorithm must be capable of rapid adaptation. We propose an algorithm which incorporates detection of the desired speech in the time-frequency domain, and employs this information to adaptively update estimates of the noise statistics. Speech detection plays a key role in ensuring the quality of the output signal. We conduct controlled measurements of the array in noisy scenarios. The proposed algorithm preforms favorably with respect to conventional algorithms.
BibTeX
@inproceedings{levin2016near,
  author = {Dovid Y. Levin and Emanuël Anco Peter Habets and Sharon Gannot},
  title = {{Near-field signal acquisition for smartglasses using two acoustic vector-sensors}},
  booktitle = {Speech Commun},
  year = {2016},
  url = {https://openreview.net/forum?id=As0tj6StLI},
}
Figure