Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract)

Ritwik Bose, Siddharth Vashishtha, James F. AllenAAAI 2020 · DOI

This research, presented as a student abstract at AAAI-20, explores a hybrid approach to natural language understanding. By combining the structural depth of logical semantic parsers with the high-accuracy "best guesses" of statistical word sense disambiguation (WSD) systems, the authors aim to overcome traditional limitations in lexical precision.

The Challenge: Lexical Ambiguity vs. Structural Depth

Semantic parsing involves translating human language into machine-readable logical forms, typically represented as graphs where nodes are word senses and edges define semantic roles. Historically, researchers have faced a trade-off between two methodologies:

  1. Logical/Rule-based Parsers (e.g., TRIPS): These systems excel at capturing complex sentence structures and supporting deep reasoning. However, they often struggle with lexical ambiguity, potentially selecting the wrong sense for a word (e.g., confusing "bank" as a financial institution with a river bank) despite understanding the grammar perfectly.
  2. Statistical WSD Systems (e.g., SupWSD): These systems are highly effective at disambiguating word senses based on context but do not produce the rich, structural logical forms required for complex AI reasoning tasks.

The authors address this by feeding the statistical insights of WSD systems into the rigid framework of a logical parser without compromising the parser's internal consistency.

Methodology: Hinting Mechanisms

The researchers integrated the SupWSD system (an SVM-based classifier) into the TRIPS parser (a bottom-up chart parser). To bridge the gap between SupWSD’s WordNet senses and the unique TRIPS ontology, a mapping was established. Two novel "hinting" mechanisms were introduced to guide the parser:

  • Prehinting: At the start of the parsing process, the parser is "primed" with specific word senses suggested by the statistical system. These senses receive a high initial score within the parser's internal workspace (the chart).
  • Progressive Hinting: As the parser builds sentence structures incrementally, it continuously checks its progress against the WSD advice. If a partial structure aligns with a suggested sense, its score is boosted, helping correct senses survive the pruning process.

Crucially, this is a "soft" guidance system. If a statistical suggestion is grammatically impossible, the parser’s internal logic retains the authority to reject it.

Key Results and Evaluation

The system was evaluated using the SemEval2013 (Task 12) dataset. Beyond standard accuracy, the authors introduced Semantic Factor Agreement (SemFac) to evaluate structural errors. This metric determines if a misclassification would actually change the overall structure of the logical form or merely mislabel a node.

Performance comparison across different parsing strategies.

As shown in the results table, the "Plain" TRIPS parser achieved an accuracy of 39.42%. By integrating both prehinting and progressive hinting (the "Comb" model), accuracy rose to 53.19%, representing a nearly 14% improvement. While the standalone SupWSD system maintains higher raw accuracy (66.19%), it lacks the ability to generate logical graphs. The hybrid approach significantly closes this gap, making logical parsers more viable for real-world applications.

Insights and Future Work

A key finding was the "Safety Net" effect: the parser proved robust against incorrect advice from the statistical system. Because the parser relies on its own grammar, it can ignore "noisy" hints that do not fit the structural context.

Future work aims to automate the mapping between WordNet and the TRIPS ontology and to incorporate richer statistical advice, such as Semantic Role Labeling and Named Entity Recognition, to further refine the parsing process.

Summary updated

BibTeX
@inproceedings{bose2020improving,
  author = {Ritwik Bose and Siddharth Vashishtha and James F. Allen},
  title = {{Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7150},
}
Figure