Building lexical semantic representations for Natural Language instructions

Elena Terenzi, Barbara Di EugenioNAACL 2003

TL;DR

This paper presents a method for automatically building a corpus of instructional text annotated with detailed lexical semantics. By integrating two rich lexical resources, VerbNet for verbs and CoreLex for nouns, with the LCFLEX parser, they achieve high accuracy in extracting semantic representations, paving the way for more robust natural language understanding of instructions.

What Problem Does This Paper Address?

Understanding natural language instructions, especially in domains like robotics or procedural guides, requires more than just parsing sentences. It necessitates capturing the underlying meaning and actions described. Specifically, this paper tackles the challenge of automatically deriving "domain knowledge" from instructional text in the form of linguistically motivated action schemes. This involves not only identifying the actions but also understanding their semantic components, like preconditions and effects. Existing methods often struggle with the nuance of verb semantics, where seemingly similar verbs can have different implications for how they interact with objects and locations. For example, "wipe the fingerprints from the counter" implies the counter is now clean, but "remove the bag" doesn't necessarily imply the bag is clean. The authors aim to build a system that can distinguish these subtle semantic differences automatically.

Methods

The core of this work lies in integrating existing lexical resources with a parser to create a system for semantic annotation.

  • Parser: They use LCFLEX, a robust left-corner parser known for its ability to handle ungrammaticalities and unknown words, which is crucial for large, real-world corpora.
  • Lexical Resources:
    • VerbNet: This resource provides a hierarchical classification of verbs based on their syntactic behavior and semantic roles, operationalizing Levin's work on verb classes. It offers detailed information on verb frames and their semantic interpretations.
  • CoreLex: This resource focuses on nouns, providing a decompositional meaning representation based on systematic polysemous classes (SPCs).
  • Ontology: VerbNet classes and CoreLex SPCs are realized as entities within a custom ontology. This ontology acts as a bridge, linking the syntactic structures parsed by LCFLEX to the semantic representations derived from VerbNet and CoreLex.
  • Integration: The LCFLEX parser's lexicon is augmented with entries derived from VerbNet and CoreLex. The parser's output, which includes syntactic roles, is then mapped to the thematic roles defined in the VerbNet class and the semantic types from CoreLex, guided by the ontology.

Datasets

  • Training/Evaluation Data: The authors evaluated their system on a test set extracted from the "home repair" portion of a 9MB written corpus originally collected at the University of Brighton.
  • Corpus Selection: They specifically collected 480 sentences that contained at least one of the verbs in their lexicon. To focus on the core action, they simplified these sentences by making the clause containing the target verb the main clause and discarding others.

Key Insights and Ideas

  • Successful Integration of Diverse Lexical Resources: The paper demonstrates that VerbNet (verb-centric, Levin's theory) and CoreLex (noun-centric, Pustejovsky's theory) can be effectively integrated into a single semantic representation framework. This is significant because these resources are based on different theoretical underpinnings.
  • Parser Augmentation for Rich Semantics: By augmenting a robust parser (LCFlex) with detailed lexical semantics from VerbNet and CoreLex, they can automatically generate semantically annotated instructional text. This moves beyond simple syntactic parsing to capture deeper meaning.
  • Importance of Noun Semantics: The evaluation results highlight the critical role of a comprehensive noun lexicon. Including CoreLex significantly improved parsing accuracy compared to using only a small, pre-defined noun lexicon.
  • Decompositional Semantics for Instructions: The work validates the utility of decompositional theories of lexical semantics for understanding the nuances of instructional language, particularly for verbs that describe actions with specific effects on objects and locations.

Key Results

The paper reports strong parsing results on instructional text:

  • Overall Accuracy: In the final evaluation (Verbs, Nouns), the system achieved 96% correct and 4% partially correct parses, with 0% parser errors.
  • Impact of Noun Lexicon: An earlier evaluation using only verbs and a small noun lexicon (Only Verbs) yielded 87% correct parses, with 4.8% partially correct and 2.2% errors. The inclusion of the full noun lexicon (CoreLex) led to a substantial improvement.

Limitations and Future Work

  • Limitations:
    • The evaluation was performed on a subset of the home repair corpus (480 sentences).
  • The authors acknowledge that further additions to the lexicon and ontology might be necessary as they scale up to the full corpus.
  • Future Work:
    • The authors plan to systematically run the parser on the entire 6MB home repair corpus.
  • They anticipate needing to expand their lexicon and ontology to cover the broader range of language encountered in the full corpus.

Summary updated

BibTeX
@inproceedings{terenzi2003building,
  author = {Elena Terenzi and Barbara Di Eugenio},
  title = {{Building lexical semantic representations for Natural Language instructions}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2003},
}
Figure