TL;DR
This paper introduces WASP, a novel statistical approach to semantic parsing that leverages state-of-the-art statistical machine translation (SMT) techniques. By framing semantic parsing as a translation problem from natural language to a formal meaning representation (MR), WASP achieves competitive accuracy and robustness, even with limited supervision.
What Problem Does This Paper Address?
Natural language understanding (NLU) has seen significant progress in shallow tasks like semantic role labeling. However, a more ambitious goal is semantic parsing: constructing a complete, formal, symbolic meaning representation (MR) of a sentence. This is crucial for applications like natural language interfaces to databases or for instructing intelligent agents. Previous methods often relied on deterministic parsing, which lacks robustness, or required extensive human effort for annotation or prior knowledge of NL syntax. The challenge is to learn a robust semantic parser that can handle complex, nested MRs with minimal supervision.
Methods
WASP's core innovation is treating semantic parsing as a statistical machine translation problem. The approach involves two main stages:
- Lexical Acquisition: This stage uses a statistical word alignment model (specifically, GIZA++ implementing IBM Model 5) to learn a bilingual lexicon. This lexicon maps natural language (NL) substrings to their translations in the target meaning representation language (MRL). The process is guided by the MRL's grammar to ensure syntactically well-formed MR translations. A key challenge addressed here is handling MR tokens that are semantically vacuous (like punctuation) or exhibit polysemy. The paper proposes a method to extract transformation rules by aligning NL words with productions in the MRL parse tree, preserving phrasal coherence.
- Probabilistic Model for Semantic Parsing: This stage uses a synchronous context-free grammar (SCFG) to define the translation process. The parser learns an SCFG lexicon and a probabilistic model that assigns likelihoods to possible derivations. The goal is to find the most probable derivation that maps an input NL sentence to its correct MR. The model is a maximum-entropy model, and parameter estimation is performed using an unsupervised approach (since gold-standard derivations are not available).
Datasets
The paper evaluates WASP on two domains:
- ROBOCUP: A corpus of 300 coach advice sentences, manually translated into English. Average sentence length: 22.52.
- GEOQUERY: A corpus of 880 English questions for querying a US geography database, manually translated into a functional, variable-free query language. Average sentence length: 7.48.
- A multilingual subset of GEOQUERY was also created, with 250 queries translated into Spanish, Japanese, and Turkish to test robustness to word order variations.
Key Insights and Ideas
- Framing Semantic Parsing as SMT: The paper demonstrates that successful SMT techniques, particularly word alignment and SCFGs, can be effectively adapted to the semantic parsing task. This opens up a new avenue for leveraging existing SMT research.
- Leveraging MRL Grammar for Lexical Acquisition: By incorporating the target MRL's grammar into the lexical induction process, WASP ensures that learned translations are syntactically valid, a significant improvement over methods that ignore MRL structure.
- Handling Ambiguity and Polysemy: The proposed method for word alignment and rule extraction effectively addresses challenges posed by semantically vacuous MR tokens and polysemous predicates, which are common issues in semantic parsing.
- Unsupervised Learning for MRs: The ability to learn a probabilistic model for derivations in an unsupervised manner is crucial, as obtaining gold-standard derivations for training is often infeasible.
Key Results
- WASP performs favorably in terms of accuracy and coverage compared to existing learning methods requiring similar amounts of supervision.
- The approach shows better robustness to variations in task complexity and word order.
- In experiments, WASP outperforms SILT (which uses a local bottom-up search for lexical learning) in recall, highlighting the effectiveness of the word-alignment-based approach.
- Performance on the multilingual GEOQUERY dataset demonstrates consistency across languages with different word orders (English/Spanish SVO vs. Japanese/Turkish SOV).
- Learning time for WASP was reported as 47 minutes for ROBOCUP and 83 minutes for GEOQUERY using standard 10-fold cross-validation.
Limitations and Future Work
- Phrasal Coherence: The paper acknowledges that the current greedy algorithm for removing links that destroy phrasal coherence is effective but suggests that a more principled way of promoting phrasal coherence is desirable.
- Awareness of MRL Syntax: The authors note that current word-based alignment models lack awareness of the MRL's tree structure, which limits the learning of better lexicons. Future work aims to develop alignment models that are aware of this structure.
- Unsupervised Learning: While unsupervised learning for the probabilistic model is a strength, the paper mentions that correct derivations are treated as hidden variables, and the Viterbi approximation is used to improve accuracy. Further exploration of unsupervised learning techniques could be beneficial.
Summary updated