For a robot to operate effectively alongside humans, it must be able to hear. Audition enables a robot to localize speakers, recognize spoken commands, detect environmental collisions, and even identify materials through touch or exploration. However, robots face a fundamental acoustic challenge that humans and stationary smart devices do not: they are incredibly loud.
When a robot moves its joints, spins its cooling fans, or steps onto a hard floor, it generates significant internal mechanical sounds known as ego-noise. Because the robot's microphones are physically proximate to its own actuators, this self-generated noise is captured at ultra-close range, creating a severe signal-to-noise ratio (SNR) bottleneck that masks external sounds. Overcoming this internal interference requires a combination of hardware-aware control strategies and adaptive signal processing.
The Acoustic Bottleneck of Robotic Proximity
In standard audio processing, background noise is typically assumed to be distant or diffuse. In robotics, however, the noise source is inside the machine itself. As detailed by Yihui Fu et al. in their work on the IEEE SLT 2021 Alpha-Mini Speech Challenge: Open Datasets, Tracks, Rules and Baselines, this internal mechanical noise creates a highly localized, intense interference profile that standard speech datasets completely ignore.
The proximity of onboard microphones to internal hardware means that cooling fans and drive motors establish a high, persistent noise floor. Mohammad MohammadAmini et al. demonstrate in RoboVox: A Single/Multi-channel Far-field Speaker Recognition Benchmark for a Mobile Robot that this constant noise floor drastically degrades the SNR directly at the microphone source, making far-field speaker recognition exceptionally difficult. In extreme cases, such as aerial robotics, the problem escalates further; Antoine Deleforge et al. note in Audio-Based Search and Rescue with a Drone: Highlights from the IEEE Signal Processing Cup 2019 Student Competition that high-speed rotors and wind resistance can bury target signals under non-stationary, harmonic noise, yielding SNRs as low as -20 dB.
[External Sound Source] ---> (Acoustic Environment) ---> [Onboard Microphone]
^
[Internal Actuators / Fans] --(Ultra-Close Proximity)-----------|
This hardware-induced interference disrupts a wide array of robotic capabilities:
- Localization and Interaction: Internal noise degrades a robot's ability to calculate sound source localization accurately. Omar Eldardeer et al. show in A Biological Inspired Cognitive Framework for Memory-Based Multi-Sensory Joint Attention in Human-Robot Interactive Tasks that this creates a major performance gap when visual cues are missing. It also masks subtle emotional cues in human speech (Egor Lakomkin et al.) and complicates multi-speaker tracking (Weipeng He et al.).
- Physical Sensing and Manipulation: When robots use acoustic feedback to detect environmental contact, ego-noise acts as a primary disruptor. For instance, Jessie van Dam et al. observe in Collision detection and identification for a legged manipulator that locomotion vibrations can easily exceed the magnitude of light environmental impacts. Similarly, Moon Young Lee notes in SonicBoom: Contact Localization Using Array of Microphones that continuous motor noise masks external collision signals. This interference even impacts manipulation tasks; Zeyi Liu et al. demonstrate in ManiWAV: Learning Robot Manipulation from In-the-Wild Audio-Visual Data that motor ego-noise creates an acoustic mismatch that can cause imitation learning policies trained on silent human demonstrations to fail.
- Human Acceptance: Beyond technical performance, ego-noise impacts how humans perceive machines. Aimee Allen et al. explore these "consequential sounds" in Robots Have Been Seen and Not Heard: Effects of Consequential Sounds on Human-Perception of Robots and Sound Judgment: Properties of Consequential Sounds Affecting Human-Perception of Robots, finding that mechanical whirrs and clicks significantly increase human anxiety and distraction. However, their findings also reveal that complete silence is undesirable, as humans rely on rhythmic, natural auditory feedback for situational awareness to track a robot's actions without looking.
Hardware and Control-Level Noise Reduction
One intuitive way to solve the ego-noise problem is to prevent it from happening at the source. For legged and domestic robots, this has led to the development of quiet locomotion strategies driven by reinforcement learning.
The primary source of high-amplitude noise in legged robots is the transient impact of feet striking the ground. To mitigate this, the authors of Minimizing Acoustic Noise: Enhancing Quiet Locomotion for Quadruped Robots in Indoor Applications developed a control strategy called MUTE (Minimizing acoUsTic noisE). By optimizing gait patterns and explicitly minimizing foot-ground impact velocity, the framework reduces locomotion noise by an average of 8 dBA. Similarly, Yuyou Zhang et al. propose a "Conditional Noise-Constrained Policy" in QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment, allowing a robot to dynamically transition into a low-noise "stealth mode" when entering sensitive environments like hospitals or offices.
However, softening foot impacts reveals a deeper problem: the mechanical noise floor. As Ryo Watanabe et al. point out in Learning Quiet Walking for a Small Home Robot, even if footstep impacts are entirely eliminated, secondary acoustic disturbances such as high-frequency motor whine and internal joint friction persist. Because software-only or control-only optimization cannot completely silence these internal actuators, algorithmic suppression remains essential.
Algorithmic Suppression and Signal Isolation
When hardware cannot be silenced, software must isolate the target signals. Modern approaches achieve this through predictive fingerprinting, spectral subtraction, and deep learning architectures.
Predictive Fingerprinting and Adaptive Filtering
Because a robot has direct access to its own motor commands and joint states, its internal noise is highly predictable. This insight allows for personalized, adaptive noise cancellation. Rodolphe Gelin et al. describe this in Audition: the most important sense for humanoid robots?, where a robot learns to identify and subtract its own unique mechanical signature from the incoming audio stream.
To handle both predictable internal sounds and unpredictable environmental noise simultaneously, Huajian Fang et al. developed a partially adaptive framework in Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise. This system pre-learns the spatial and spectral "fingerprint" of the robot's joints, utilizing prior mechanical knowledge to eliminate ego-noise while remaining flexible enough to adapt to changing external surroundings.
Inter-Frame Spectral Subtraction
For continuous, stationary internal noise—such as the steady hum of a cooling fan—algorithmic approaches can exploit temporal differences between the noise floor and non-stationary target signals like human speech. In Estimation of the Direct-Path Relative Transfer Function for Supervised Sound-Source Localization, Xiaofei Li et al. utilize an inter-frame spectral subtraction technique. By subtracting the stationary noise energy calculated across successive frames, the algorithm isolates the direct-path acoustic transfer function of external sounds without requiring a dedicated, pre-trained noise estimation model.
Deep Learning Isolation
When a robot is actively moving, its ego-noise becomes highly dynamic and non-linear, rendering traditional linear filters insufficient. Modern systems rely on deep neural networks to disentangle these overlapping acoustic streams.
[Mixed Audio Input] ---> [ Deep Neural Network ] ---> [Isolated Target Signal]
(Speech + Ego-Noise) (Learns non-linear (Clean Speech or
motor/joint patterns) Contact Sound)
In interactive perception tasks, such as shaking a container to identify its contents, the sound of the object is tightly coupled with the sound of the robot's own hand. Manfred Eppe et al. demonstrate in Deep Neural Object Analysis by Interactive Auditory Exploration with a Humanoid Robot that recurrent neural network (RNN) architectures can successfully learn the temporal patterns of the robot's active exploration, allowing the network to isolate task-relevant object sounds from internal motor noise.
This capacity for deep isolation also enables continuous tracking during locomotion. In The Un-Kidnappable Robot: Acoustic Localization of Sneaking People, Mengyu Yang et al. show that deep learning models can isolate incredibly weak external signals—such as the rustle of clothing or a soft footstep—from the high-magnitude self-noise of a moving robot. This allows the system to maintain continuous auditory tracking without requiring the robot to stop and listen, bridging the gap between hardware limitations and robust environmental perception.
Key Foundational Work
Active Audition for Humanoid (2000) — contributed 2 concept elements to this direction.