Abstract
Semantic parsers are critical components of intelligent personal agents, translating natural language into actionable logical forms. However, these parsers often struggle with out-of-vocabulary (OOV) words and the "catastrophic forgetting" that occurs during retraining. This paper introduces novel neural architectures, including an Argument Transfer model and a hybrid paraphrase generator, designed to enhance robustness to OOV terms while maintaining performance on previously learned instructions. Experimental results on benchmark datasets demonstrate significant improvements in OOV handling and stability during continual learning.
Introduction
Intelligent agents like Siri or Alexa rely on semantic parsing to interpret user commands. A significant challenge in this domain is the limited vocabulary of training datasets compared to real-world usage. For example, the ATIS dataset contains fewer than 1,000 words, whereas domain-specific reviews can exceed 100,000 words. When users employ synonyms or new terminology (OOV words), standard parsers often fail. Furthermore, fine-tuning models on new data frequently degrades performance on original tasks. This research addresses the dual goals of runtime robustness to OOV words and stable continual learning.
Methodology
The researchers propose three primary enhancements to standard Sequence-to-Sequence (Seq2Seq) and Sequence-to-Tree models:
Robust Embeddings
Instead of random initialization for unknown tokens, the models utilize pre-trained word embeddings (e.g., GloVe). This enables the parser to leverage semantic similarities between OOV words (like "large") and known vocabulary (like "big").
Argument Transfer Model
The Argument Transfer decoder incorporates a neural classifier to predict when an "argument" (e.g., a city name or date) should be output. Upon detection, a binary lookup vector restricts output choices to specific arguments present in the input query, preventing confusion from unknown entities.
Hybrid Paraphrase Generation
To handle OOV-heavy queries at runtime, a modular "translator" is placed before the parser. If the base parser's confidence score falls below a threshold, the query is routed to a paraphrase generator. This component converts OOV terms into the parser's known vocabulary before final processing.
Experimental Setup
The models were evaluated on three benchmark datasets: GEO (US geography), JOB (job listings), and ATIS (airline reservations). To specifically test OOV performance, the authors developed PARA-GEO, PARA-JOB, and PARA-ATIS by substituting words in the original test sets with synonyms from WordNet.
Results and Discussion
The proposed models achieved state-of-the-art (SOTA) accuracy on standard benchmarks, reaching 88.3% on GEO and 85.27% on ATIS.
OOV Resilience
On the PARA-GEO dataset, the baseline model's accuracy was 53.6%. The combination of paraphrase generation and argument transfer significantly improved this to 65.61%.
Stability in Continual Learning
A key finding was the mitigation of the "fine-tuning trap." While baseline models often saw accuracy drops of up to 2% on original tasks after retraining for new words, the hybrid paraphrase model demonstrated remarkable stability. In the ATIS dataset, it achieved a 0.08% gain in overall accuracy after learning new terms, indicating effective continual learning without forgetting.
Conclusion
This work presents a robust framework for semantic parsing in the presence of OOV words. By combining pre-trained embeddings, constrained decoding via argument transfer, and modular paraphrasing, the authors demonstrate that intelligent agents can expand their vocabulary without sacrificing existing capabilities. Future research will explore more complex structural paraphrasing and long-term sequential learning.
Summary updated