The EAR Project

Julien Bonnal, Sylvain Argentieri, Patrick Danès, Jérôme Manhès, Philippe Souères, Marc RenaudJournal of the Robotics Society of Japan 2010 · DOI

Abstract

The EAR (Embedded Audition for Robotics) Project addresses the critical need for high-performance, real-time auditory processing in mobile robotics. This paper details the development of a compact, low-power (<500mW) embedded sensor system utilizing an 8-microphone array and FPGA-based hardware-software co-design. By implementing complex acoustic algorithms—including MUSIC for source localization and frequency-invariant beamforming for sound isolation—directly onto a Xilinx Virtex-4 FPGA, the system provides a robust auditory front-end that significantly reduces the computational burden on a robot's main processor.

Introduction

While computer vision is a standard component in robotics, "audition" (hearing) remains a significant challenge. Effective robotic hearing must overcome high computational loads for multi-channel processing, strict physical and power constraints for mobile deployment, and the acoustic complexity of real-world environments characterized by reverberation and wideband noise. The EAR project bridges the gap between theoretical Computational Auditory Scene Analysis (CASA) and practical deployment by offloading signal processing to dedicated hardware.

System Architecture and Methodology

The EAR system consists of a uniform linear array of eight electret microphones, a custom signal conditioning unit, and a Data Acquisition (DAQ) board equipped with 18-bit ADCs. At its core is a Xilinx Virtex-4 FPGA, which enables massive parallel processing of tasks such as Fast Fourier Transforms (FFTs) and spatial filtering with minimal latency.

The EAR system architecture, detailing the interaction between the FPGA board and the DAQ board.

The hardware-software co-design utilizes advanced algorithms to ensure performance:

  • Frequency-Invariant Beamforming: Uses modal analysis and convex optimization to maintain a consistent "listening" pattern across the 300 Hz to 3 kHz range, which is essential for human speech.
  • MUSIC (MUltiple SIgnal Classification): A beamspace implementation of MUSIC is used for high-resolution Direction of Arrival (DOA) estimation, reducing computational costs for simultaneous source localization.
  • MAICE: The Minimum Akaike Information Criterion is employed to automatically detect the number of active sound sources.

By offloading these tasks, the robot's main processor (the UNIX host) is freed for higher-level social behaviors and speech recognition.

Experimental Results

The system was evaluated using both simulated data and real-world laboratory experiments involving a microphone array mounted on a mast and mobile targets. Key performance metrics include:

  • Localization Accuracy: Azimuth estimation error remained below ±10° in experimental tests.
  • Multi-Source Capability: The system successfully localized up to three simultaneous sound sources using four beamspaces.
  • Computational Efficiency: The optimized farfield pseudo-spectra computation achieved a 15x speedup compared to standard broadband MUSIC implementations.
  • Power Consumption: The entire 8-channel processing pipeline operates at under 500mW, a critical threshold for battery-operated mobile robots.
Experimental results showing polar plots of beamforming patterns and source localization performance.

Conclusion and Future Work

The EAR Project demonstrates a viable path for integrating sophisticated auditory capabilities into mobile robots without compromising battery life or primary processing power. Future work focuses on fully hardcoding the MUSIC algorithm into the FPGA using Jacobi and CORDIC algorithms and testing the system in high-noise public environments to further validate its robustness in real-world social robotics applications.

Summary updated

BibTeX
@article{bonnal2010ear,
  author = {Julien Bonnal and Sylvain Argentieri and Patrick Danès and Jérôme Manhès and Philippe Souères and Marc Renaud},
  title = {{The EAR Project}},
  journal = {Journal of the Robotics Society of Japan},
  year = {2010},
  doi = {10.7210/jrsj.28.10},
}
Figure