Coping with Extragrammaticality

Jaime G. Carbonell, Philip J. HayesCOLING 1984

TL;DR

This paper tackles the challenge of building natural language interfaces (NLIs) that can understand and process user input that deviates from perfect grammar. The authors propose a taxonomy of "extragrammaticalities" and a toolkit of computational strategies to handle them, arguing that case frame instantiation parsers offer the most robust foundation for such forgiving interfaces.

What Problem Does This Paper Address?

Natural language interfaces, especially those designed for specific domains (like database queries or command systems), often struggle when users don't adhere to strict grammatical rules. This "extragrammaticality" can manifest in various ways: misspellings, missing words, extra words, incorrect word order, and even violations of semantic rules. Traditional parsers, built on rigid grammatical structures, tend to fail when encountering such input, leading to frustrating user experiences. The core problem this paper addresses is how to make NLIs more robust and forgiving, enabling them to understand and act upon imperfect, yet often comprehensible, user utterances.

Methods

The paper's primary contribution lies in its systematic approach to understanding and handling extragrammaticality.

  • Taxonomy of Extragrammaticalities: The authors propose a classification of grammatical deviations, categorizing them into lexical (e.g., misspellings, unknown words) and sentential (e.g., missing constituents, spurious words, out-of-order constituents, agreement violations, semantic constraint violations) levels.
  • Recovery Strategies: For each category of deviation, the paper outlines specific computational strategies. These include:
  • Lexical: Spelling correction sensitive to syntactic and semantic expectations, and methods for handling unknown words by inferring meaning from context or interacting with the user.
  • Sentential: Techniques for filling in missing words based on semantic expectations, ignoring spurious words, reordering constituents, and resolving semantic constraint violations through focused interaction or by exploiting related concepts.
  • Parsing Approach Evaluation: The authors analyze how well existing parsing paradigms (transition network parsers, pattern matching parsers, and case frame instantiation parsers) can support these recovery strategies. They argue that case frame instantiation parsers are inherently more interpretive and better equipped to handle extragrammaticality due to their ability to leverage semantic and pragmatic information and their non-uniform representation.

Datasets

Not discussed in the paper. The authors mention their strategies have been tested in existing parsers like FLEXP, CASPAR, and DYPAR, but specific datasets are not detailed.

Key Insights and Ideas

  • Extragrammaticality is System-Dependent: The paper defines extragrammaticality not as an absolute violation of language rules, but relative to the capabilities of a specific parsing system. This pragmatic view is crucial for building practical NLIs.
  • Semantic Information is Key: A recurring theme is the critical role of semantic and pragmatic knowledge in recovering from grammatical errors. Syntactic rules alone are often insufficient.
  • "Stepping Back" is Essential: Robust recovery often requires the parser to temporarily suspend strict expectations, analyze the broader context, and then re-evaluate the input. This "interpretive" ability is a desirable trait for forgiving parsers.
  • Case Frame Parsers Offer a Strong Foundation: The paper makes a compelling case for case frame instantiation as a superior parsing paradigm for handling extragrammaticality, primarily due to its inherent interpretiveness and ability to integrate semantic knowledge.
  • Multi-Strategy Approach is Ideal: While case frame instantiation is favored, the authors suggest that the ultimate solution might lie in a multi-strategy approach, combining various parsing and recovery techniques.

Key Results

The paper doesn't present quantitative results from new experiments. Instead, it offers a qualitative analysis and a framework for understanding and implementing robust parsing. The key "results" are:

  • A comprehensive taxonomy of extragrammatical phenomena.
  • A "tool chest" of computationally tractable recovery strategies.
  • An evaluation of how well different parsing approaches support these strategies, concluding that case frame instantiation is the most promising.
  • A set of desiderata for parsing processes designed to handle extragrammatical input: interpretiveness, easy integration of semantic information, ability to exploit non-uniformity, and support for both top-down and bottom-up processing.

Limitations and Future Work

  • Dialogue Level Extragrammaticality: While the paper focuses on lexical and sentential levels, it acknowledges that dialogue phenomena (like ellipses) are also a significant source of extragrammaticality and refers to other work for details.
  • Compound Errors: The paper notes that combining multiple lexical deviations (e.g., misspellings and run-on words) requires combinatorially inefficient recovery strategies, suggesting a need for further work in this area.
  • Multi-Strategy Integration: The authors believe that a multi-strategy approach, potentially integrating case frame instantiation with other techniques, offers the best path forward for truly robust NLIs. They refer to other work for a more detailed discussion of this.
  • Focused Interaction: The importance of focused clarification dialogues with the user to resolve ambiguities or confirm assumptions is highlighted, suggesting this is a crucial component of a complete recovery system.

Summary updated

BibTeX
@inproceedings{carbonell1984coping,
  author = {Jaime G. Carbonell and Philip J. Hayes},
  title = {{Coping with Extragrammaticality}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1984},
}
Figure