Abstract
Service robots operating in domestic environments must reliably interpret spoken commands, yet Automatic Speech Recognition (ASR) systems frequently produce errors due to environmental noise or non-native accents. This paper introduces Seq2Seq-NI, a method that enhances robot understanding by training an encoder-decoder neural network with phoneme-level noise injection. By simulating ASR errors during training, the system learns to map corrupted speech recognition results to accurate robot command sequences, significantly improving robustness over traditional rule-based methods.
Introduction
The standard pipeline for robot speech understanding involves capturing a speech signal, processing it through an ASR system, performing semantic parsing, and finally executing the robot action. However, the transition from ASR to semantic parsing is often a "weak link." Traditional rule-based parsers are brittle; even a single misrecognized phoneme can lead to total command failure. While ASR technology continues to improve, errors remain inevitable, necessitating a more robust approach to semantic understanding.
Methodology: Seq2Seq with Noise Injection
The authors propose a semantic parsing system based on a Sequence-to-Sequence (Seq2Seq) architecture equipped with an attention mechanism.
Architecture
The model uses an encoder-decoder structure where the encoder processes input phoneme sequences and the decoder generates robot commands. The attention mechanism allows the decoder to focus on specific parts of the input sequence during generation, which is critical for handling long or complex instructions.
Noise Injection Strategy
The core innovation is the Stochastic Deformation Model (SDM) used during the training phase. Rather than training on "perfect" text, the authors inject artificial noise into phoneme sequences through three operations:
- Insertion: Adding random phonemes.
- Deletion: Removing phonemes.
- Substitution: Swapping phonemes for others.
This phoneme-level approach is superior to word-level noise because it captures the phonological similarities common in ASR errors (e.g., "take it" becoming "tickets"). By training on this "noisy" data, the neural network learns to recover the intended meaning from garbled input.
Experimental Setup
The system was evaluated using a synthetic dataset of 10,000 training sentences generated via the GPSR Sentence Generator (based on RoboCup@Home tasks). To test real-world performance, four participants recorded test sentences in noisy domestic environments. These recordings were processed through two commercial ASR systems—Google Speech API and CMU Sphinx—to provide a realistic baseline of recognition errors.
Key Results and Insights
The Seq2Seq-NI model significantly outperformed all baselines. In tests using the Google Speech API, which suffered from a near 100% sentence error rate in noisy conditions, traditional rule-based and standard Seq2Seq models failed completely. In contrast, Seq2Seq-NI achieved a 24% success rate in perfect command reconstruction.
Beyond success rates, the model demonstrated remarkable robustness. It successfully inferred commands like Grasp(camera) even when the ASR output was nonsensical (e.g., "next detect camera tickets"). A key finding was that the optimal noise level during training (ρ) directly correlates with the expected ASR error rate during deployment, suggesting that noise injection acts as a specialized form of data augmentation for speech robustness.
Conclusion and Future Work
Seq2Seq-NI provides a robust framework for robot command understanding in the face of inevitable ASR errors. However, the current system operates on a closed vocabulary of 110 lexicons. Future research will focus on handling "Out of Vocabulary" words, implementing on-line learning for new objects, and integrating environmental context—such as visual data—to further improve the robot's interpretive accuracy.
Summary updated