Abstract
Existing studies on semantic parsing mainly focus on the in-domain setting. We formulate cross-domain semantic parsing as a domain adaptation problem: train a semantic parser on some source domains and then adapt it to the target domain. Due to the diversity of logical forms in different domains, this problem presents unique and intriguing challenges. By converting logical forms into canonical utterances in natural language, we reduce semantic parsing to paraphrasing, and develop an attentive sequence-to-sequence paraphrase model that is general and flexible to adapt to different domains. We discover two problems, small micro variance and large macro variance, of pre-trained word embeddings that hinder their direct use in neural networks, and propose standardization techniques as a remedy. On the popular Overnight dataset, which contains eight domains, we show that both cross-domain training and standardized pre-trained word embeddings can bring significant improvement.
Introduction
Semantic parsing is the task of mapping natural language utterances (e.g., "Who is Alice's boss?") into machine-readable logical forms (e.g., R[manager].employee.Alice). While significant progress has been made in in-domain settings, cross-domain adaptation remains a major hurdle. Different domains utilize distinct predicates and symbols; for instance, a model trained on basketball data (using terms like team or season) cannot easily generalize to a social network domain (using employer or friend). Furthermore, the high cost of collecting labeled data for every new domain makes zero-shot or few-shot cross-domain transfer a critical area of research.
Methodology
Semantic Parsing via Paraphrasing
The core innovation of this work is reframing semantic parsing as a paraphrasing task. Instead of mapping an input utterance directly to a complex logical form, the authors introduce an intermediate step: Canonical Utterances. These are natural-language-like representations of logical forms generated by a simple, domain-specific grammar. For example, the logical form for "Alice's employer" might be represented by the canonical utterance "employee Alice whose employer is...".
By converting the problem into a paraphrase task (mapping the input "Who does Alice work for?" to the canonical version), the model can leverage general linguistic knowledge that remains constant across domains. This "Universal Bridge" allows the model to learn the concept of relationships from source domains and apply them to new predicates in a target domain.
Attentive Seq2Seq Model
The framework utilizes an attentive sequence-to-sequence model consisting of a bi-directional Gated Recurrent Unit (GRU) encoder and an attentive GRU decoder. This architecture is designed to be general and flexible, allowing it to adapt to various domains by focusing on the linguistic similarities between input and canonical utterances.
Word Embedding Standardization
The authors identify that raw pre-trained word embeddings (like Word2Vec) often underperform in neural networks due to two statistical issues:
- Small Micro Variance: Values within individual vectors are often too small, leading to poor optimization during the initial stages of training.
- Large Macro Variance: Significant differences between vectors can hinder the model's ability to generalize to unseen words.
To address this, they propose unit-variance standardization (specifically "per-example standardization" or ES). This technique scales embeddings to have unit variance, making them more compatible with standard neural network initialization methods like Xavier initialization.
Experimental Results
The model was evaluated on the OVERNIGHT benchmark, which includes eight distinct domains: Calendar, Blocks, Housing, Restaurants, Publications, Recipes, Social, and Basketball. External linguistic knowledge was provided by Word2Vec embeddings pre-trained on the Google News corpus.
As shown in the results table, the full model (WORD2VEC + ES + X) achieved a state-of-the-art average accuracy of 80.6%, outperforming previous best-in-class models. A key finding was the impact of standardization: using the proposed ES technique led to a roughly 10% absolute improvement in accuracy compared to using raw pre-trained embeddings (comparing WORD2VEC + I to WORD2VEC + ES + I). Additionally, the model demonstrated high data efficiency; in low-data scenarios, cross-domain training allowed the model to achieve near-full performance using only 30% of target domain training data.
Conclusion
This research demonstrates that natural language serves as a more effective transfer medium than formal logic for cross-domain semantic parsing. While the approach requires a basic grammar for canonical utterances, it significantly reduces the need for extensive labeled data in new domains. Future work will explore scaling these techniques to more complex logical constructs and even more diverse domain frameworks.
Summary updated