What's in an ontology for spoken language understanding

Silvia Quarteroni, Giuseppe Riccardi, Marco DinarelliInterspeech 2009

Sure, here's a blog-style summary of the paper "What's in an Ontology for Spoken Language Understanding":

TL;DR

This paper introduces a novel approach to Spoken Language Understanding (SLU) by leveraging ontologies to model domain knowledge. By representing concepts as either predicates or arguments and using classical semantic relations, the authors demonstrate that ontology-based relatedness can significantly improve the accuracy of SLU systems, offering a more robust and shareable domain modeling solution.

What Problem Does This Paper Address?

Current Spoken Language Understanding (SLU) systems often rely on hand-crafted semantic grammars or simple attribute-value labeling. While effective for specific tasks, these approaches suffer from a lack of reusability and portability. Building a new domain model from scratch for each new application is time-consuming and inefficient. Furthermore, existing models often fail to capture the rich relationships between concepts, limiting their ability to understand nuanced language. This paper addresses the need for a more flexible, reusable, and semantically grounded approach to domain modeling in SLU.

Methods

The authors propose using an ontology as a domain model for SLU. Their approach involves:

  • Ontology Design: Concepts are classified as either predicates (actions) or arguments (objects), mirroring predicate-argument structures found in linguistic theories like FrameNet. This creates a hierarchical structure rooted in an abstract Concept class.
  • Classical Semantic Relations: The ontology incorporates classical semantic relations (e.g., IS-A, SUPER, REL, NEGATE) to define how concepts relate to each other. This allows for a richer representation of domain knowledge than simple attribute-value pairs.
  • Ontology Relatedness Metric: A binary relatedness measure r(c1, c2) is defined between ontology concepts. This measure is based on whether concepts share a relation or are related through their superclasses. A combined utterance-level relatedness metric relw(hyp) is then calculated for SLU hypotheses.
  • Lightweight Implementation: The ontology is designed to be lightweight and easily integrated into existing SLU systems. They demonstrate this by mapping Protégé ontology files to Java packages, enabling seamless updates.

Datasets

The authors used the LUNA dataset, a corpus of Human-Machine (HM) dialogs recorded in the context of a technical support center. They acquired and manually transcribed 4500 HM turns, which were then annotated using their ontology. The test set used for evaluation consisted of 128 dialogs with 585 turns and 599 non-null concepts.

Key Insights and Ideas

  • Ontologies as a Bridge: Ontologies provide a structured and reusable way to represent domain knowledge, overcoming the limitations of ad-hoc grammars.
  • Predicate-Argument Structure is Key: Classifying concepts as predicates or arguments offers a natural way to model actions and the entities they affect, aligning well with linguistic semantics.
  • Classical Relations Matter: Incorporating established semantic relations allows for a deeper understanding of how concepts interact, leading to more accurate interpretations.
  • Relatedness as a Confidence Signal: The ontology-relatedness metric can serve as a valuable signal for validating SLU hypotheses, indicating how plausible an interpretation is within the domain context.
  • Lightweight Integration is Feasible: The proposed approach is designed to be easily integrated into existing SLU pipelines, making it practical for real-world applications.

Key Results

The paper demonstrates a positive correlation between the ontology-relatedness of SLU hypotheses and the probability of correct interpretation. Specifically:

  • The probability of correct interpretation tends to increase as the ontology-relatedness score increases.
  • This trend is observed across different ranks of SLU hypotheses, suggesting the relatedness metric is a robust indicator of correctness.
  • An initial experiment with ontology-based reranking showed promising results, indicating that interpretations with higher relatedness are more likely to be correct.

Limitations and Future Work

  • Relatedness vs. Correctness: The authors acknowledge that ontology relatedness is not a direct measure of correctness. Some correct interpretations may not exhibit high relatedness, as concepts in spontaneous speech don't always need to be explicitly related.
  • Confidence Metric Conversion: They are currently studying how to convert the ontology-relatedness into a suitable confidence metric that can be combined with the existing confidence scores from their statistical SLU model.
  • Reranking Refinement: Further work is needed to refine the ontology-based reranking strategy to effectively leverage the relatedness metric.
  • Broader Domain Application: While the approach is designed to be generic, further experiments on different domains would validate its broader applicability.

Summary updated

BibTeX
@inproceedings{quarteroni2009what,
  author = {Silvia Quarteroni and Giuseppe Riccardi and Marco Dinarelli},
  title = {{What's in an ontology for spoken language understanding}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure