Unsupervised Semantic Parsing

Hoifung Poon, Pedro M. Domingos

Introduction

Semantic parsing—the process of converting natural language into formal, machine-executable logical representations—is a cornerstone of deep language understanding. Traditionally, this task has been treated as a supervised learning problem, requiring extensive "gold standard" datasets where sentences are manually paired with logical formulas. However, the cost and lack of scalability of manual labeling have limited the application of semantic parsers to narrow domains.

This paper introduces USP (Unsupervised Semantic Parsing), the first system capable of inducing semantic parsers from raw, unlabeled text. By leveraging Markov logic to cluster syntactic variations of the same meaning, USP extracts knowledge and answers complex questions with performance that significantly exceeds previous open-domain information extraction systems.

The Challenge of Semantic Variation

Natural language is inherently redundant and ambiguous. A single underlying meaning can be expressed through a multitude of syntactic structures, word choices, and voices (e.g., active vs. passive). For instance, "X regulates Y" and "Y is regulated by X" represent the same semantic relationship despite different surface forms.

While existing unsupervised tools like TextRunner or DIRT can identify shallow triples or paraphrases, they struggle to produce a complete, recursive formal meaning representation. USP aims to bridge this gap by learning a "language of meaning" directly from raw text, allowing for deeper reasoning and more robust question answering.

The USP Methodology

The USP framework is built upon three foundational pillars designed to transform raw syntax into structured semantics.

1. Syntax-to-Semantics Pipeline

Rather than attempting to learn syntax and semantics simultaneously, USP decouples the two. It utilizes an off-the-shelf dependency parser to process raw text into dependency trees. These trees are then converted into Quasi-Logical Forms (QLFs)—sets of logical atoms that represent words and their grammatical relationships. This approach allows the system to focus entirely on the translation of syntactic structures into semantic ones.

2. Relational Clustering

The core of USP is its ability to cluster lambda-form expressions that share the same meaning. This process is recursive: if two expressions have sub-parts that belong to the same semantic clusters, the expressions themselves are likely to be semantically equivalent. For example, USP might learn that the cluster for "borders" includes variations like "is next to" or "shares a border with" because they consistently appear with similar arguments.

3. Markov Logic Networks (MLNs)

To handle the inherent ambiguity of language, USP employs Markov logic to define a joint probability distribution over logical forms and their cluster assignments. This probabilistic grounding allows the system to make globally consistent decisions about meaning while accounting for uncertainty.

The Learning Process

USP induces its parser through a greedy search-based approach. The system iteratively refines its semantic clusters using two primary operators:

  • Merge: This operator combines two clusters if they appear to represent the same underlying meaning.
  • Compose: This operator creates larger, more complex clusters by combining existing ones that frequently appear together in a specific relationship.

By maximizing the likelihood of the observed text under the MLN model, USP discovers the most parsimonious and accurate mapping from syntax to logic.

Empirical Evaluation

The authors evaluated USP on the GENIA dataset, which consists of 1,999 PubMed biomedical abstracts. The model was trained on the raw text and then tested on a benchmark of 2,000 questions (e.g., "What regulates MIP-1alpha?") derived from the most frequent verbs and entities in the corpus.

USP was compared against several baselines, including TextRunner (an open IE system), DIRT (a paraphrase discovery system), and an informed keyword-based search baseline.

Key Results

USP demonstrated a dramatic improvement over the baselines in both precision and recall:

  • Accuracy: USP achieved 88% accuracy, significantly higher than DIRT (59%) and TextRunner (53%).
  • Recall: USP provided 295 correct answers, more than tripling the output of the next best system (DIRT at 94 and TextRunner at 81).

Qualitative analysis confirmed that USP successfully identified clusters of synonymous verbs, such as {investigate, examine, evaluate, analyze, study, assay} and {diminish, reduce, decrease, attenuate}, without any prior knowledge of these relationships.

Limitations and Future Work

While USP represents a significant leap forward, several challenges remain. The system occasionally clusters antonyms (e.g., "mature" and "immature") because they appear in similar contexts. Additionally, the current version does not handle complex logical phenomena like quantifier scoping or discourse-level coreference across multiple sentences. Future research will focus on scaling USP to web-scale datasets and incorporating more sophisticated logical reasoning capabilities.

Conclusion

USP proves that it is possible to learn deep semantic representations without the need for expensive labeled data. By treating semantic parsing as a recursive clustering problem within a probabilistic framework, USP opens new avenues for automated knowledge extraction and natural language understanding across diverse domains.

Summary updated

Abstract

We present the first unsupervised approach to the problem of learning a semantic parser, using Markov logic. Our USP system transforms dependency trees into quasi-logical forms, recursively induces lambda forms from these, and clusters them to abstract away syntactic variations of the same meaning. The MAP semantic parse of a sentence is obtained by recursively assigning its parts to lambda-form clusters and composing them. We evaluate our approach by using it to extract a knowledge base from biomedical abstracts and answer questions. USP substantially outperforms TextRunner, DIRT and an informed baseline on both precision and recall on this task.
BibTeX
@inproceedings{poon2009unsupervised,
  author = {Hoifung Poon and Pedro M. Domingos},
  title = {{Unsupervised Semantic Parsing}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2009},
}
Figure