A Rule-Based Approach to Ill-Formed Input

Norman K. Sondheimer, Ralph M. WeischedelCOLING 1980

TL;DR

This paper proposes a novel framework for natural language interfaces to handle "ill-formed" input (misspellings, ungrammaticality, etc.) by treating it as rule-based. Instead of rejecting such input, the system uses "meta-rules" to modify normal processing rules, allowing for intelligent guessing and clarification, ultimately making interfaces more robust and user-friendly.

What Problem Does This Paper Address?

Natural language interfaces (NLIs) have made significant strides, but they often struggle with the messy reality of human language: ill-formed input. This includes everything from simple typos and grammatical errors to more complex issues like ambiguous phrasing or requests that exceed the system's capabilities. While some systems adapt by expecting users to conform to strict input formats, this paper argues that this defeats a primary purpose of NLIs – making systems accessible to infrequent users without extensive training. The authors highlight the need for NLIs to either intelligently infer user intent, request clarification, or at least accurately identify the nature of the ill-formedness, rather than simply failing.

Methods

The core of the paper's proposal is to treat ill-formedness as a rule-based phenomenon. This involves two types of rules:

  • Normal Processing Rules: These are the standard rules used to interpret well-formed input.
  • Meta-Rules: These are special rules that are invoked only when normal processing rules are violated. Their purpose is to modify the normal rules to accommodate ill-formed input.

The authors envision this approach impacting three phases of NLI development and use:

  1. Development Phase: NLIs would have separate components for handling well-formed and ill-formed inputs. For example, a grammar would be written for grammatically correct sentences, and a separate set of meta-rules would be developed for handling variations.
  2. Error Identification Phase: When normal processing fails, the system would analyze the violation of normal rules using the defined meta-rules. This helps pinpoint the source of the ill-formedness.
  3. Error Recovery Phase: Meta-rules would guide the modification of normal processing rules to interpret the ill-formed input as completely as possible. This could involve relaxing constraints or making educated guesses.

The paper draws an analogy to programming language compilers, which also have mechanisms for detecting and recovering from errors.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Ill-formedness as a Feature, Not a Bug: The central idea is that ill-formedness isn't just something to be avoided but a predictable aspect of language that can be managed through a structured, rule-based system.
  • Meta-Rules as the Key: The introduction of "meta-rules" is crucial. These rules don't directly process input but rather dictate how the standard processing rules should be adapted when errors occur. This allows for a more systematic and generalizable approach to error handling.
  • Separation of Concerns: The proposed approach advocates for separating the development of well-formed input processing from ill-formed input handling. This simplifies the design of each component and avoids unnecessary overhead for well-formed inputs.
  • Leveraging Linguistic Evidence: The authors draw support from linguistics, where similar concepts of rule relaxation and modification are used to explain grammatical variations and errors.
  • Cooperative User Assumption: The approach assumes a cooperative user, which helps limit the scope of ill-formedness to manageable levels (e.g., avoiding deliberately nonsensical inputs).

Key Results

The paper presents a conceptual framework and argues for its effectiveness through examples and supporting evidence from related research. It doesn't present quantitative results from a specific implemented system but rather makes a case for the approach. The authors suggest that this rule-based method can:

  • Ease system development by allowing independent design of normative and error-handling components.
  • Reduce runtime costs by only invoking ill-formedness processing when necessary.
  • Promote code reuse and generalization by describing ill-formedness through meta-rules that relate to normative rules.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Cooperative User Assumption: The approach relies on users not deliberately trying to break the system. Handling uncooperative users is outside the scope.
  • Overshoot: Users might request capabilities beyond the system's scope. The authors assume users have basic familiarity with the system's purpose and power, but detecting and handling "overshoot" requires further work.
  • Completeness of Understanding: While the goal is to interpret ill-formed input as completely as possible, the paper notes that relaxing constraints can introduce uncertainty, meaning complete understanding is not always guaranteed.

Future work includes:

  • Reformulating Existing Techniques: Integrating popular techniques for encoding ill-formed patterns into ATN grammars within their meta-rule methodology.
  • Handling Additional Classes of Ill-formedness: Developing strategies for merged thoughts/run-on sentences, wrong word choices (beyond spelling), "expansion" ellipsis (fragmentary input), and violations of semantic constraints.
  • Parallel Processing: Improving ill-formedness handling by parallel processing of lexical, semantic, syntactic, and pragmatic components, particularly for detecting pragmatic and semantic overshoot.

Summary updated

BibTeX
@inproceedings{sondheimer1980rule,
  author = {Norman K. Sondheimer and Ralph M. Weischedel},
  title = {{A Rule-Based Approach to Ill-Formed Input}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1980},
}
Figure