Variable-Depth Natural Language Understanding

Daniel Kayser, Daniel Coulon

Abstract

Traditional Natural Language Understanding (NLU) systems often rely on fixed-depth representations, treating all information with equal granularity regardless of context. This paper introduces Variable-Depth Processing (VDP), a framework designed to allow systems to adjust their depth of analysis dynamically. By employing a "plausibility calculus" and "onion-skin" rules, VDP enables progressive knowledge retrieval and more efficient resource management, mirroring human cognitive processes and challenging the necessity of universal canonical representations.

Introduction: The Problem of Fixed-Depth Representations

In early AI research, knowledge representation was typically rigid. Whether a system encountered a word in a casual sentence or as the focal point of a complex query, it applied a "fixed-depth" representation—a constant volume of information, such as a predicate calculus formula or a conceptual dependency diagram.

This approach is both computationally inefficient and cognitively unrealistic. Humans do not process every word of a newspaper article with the same intensity applied to a legal contract. Furthermore, the authors argue against the existence of "canonical representations"—the notion that there is a single, perfect way to represent the meaning of a sentence. Instead, they posit that meaning is fluid and dependent on the required depth of understanding for a specific task.

Methodology: Variable-Depth Processing (VDP)

The proposed VDP framework rests on three primary pillars:

  1. Progressive Description: Knowledge is stored hierarchically. A system might initially recognize a "Christmas tree" at a coarse level (e.g., associated with presents and family) and only deepen the analysis to include botanical details like resin or needles if the context demands it.
  2. Multiple Strategies: The system switches between "skimming" (loose semantic evaluation) and "careful reading" (rigorous logical and syntactic processing) based on goals and available computational resources.
  3. Evaluation of Approximation: To manage incomplete information, the authors propose a Plausibility Calculus. Unlike probabilistic or fuzzy logic, this calculus utilizes composition laws for weakening (accounting for missing information) and reinforcement (accounting for corroborating data).

Contextual Production Rules and the Onion Skin Model

To implement VDP, the authors suggest Contextual Production Rules (CPR). These rules are structured like an "onion skin": a core rule is surrounded by layers of conditions ($\gamma_i$) that, when verified, increase the plausibility of the interpretation.

The 'Onion Skin' structure of Contextual Production Rules, showing nested conditions and associated plausibility values.

As the system allocates more time to a task, it "peels" back the layers of the onion, checking more conditions and refining the plausibility of its results. This allows for "anytime" performance, where the system can provide its best current estimate at any point in the processing cycle.

Key Insights

  • Rejection of Canonicity: Equivalence between expressions is not a universal property but a function of the depth of analysis.
  • Triggered Deepening: Depth of analysis is not random; it is triggered by internal factors (finding an interesting coarse-level match) or external factors (typography like italics, or intonation in speech).
  • Resource Management: VDP serves as an early precursor to "anytime algorithms" in NLU, optimizing the trade-off between accuracy and computational cost.

Conclusion

While the paper is primarily theoretical, it formalizes a roadmap for AI to mimic hierarchical human retrieval. The authors acknowledge challenges in weighting factors within the plausibility calculus and the potential overhead of managing multiple strategies. However, they suggest that variable-depth processing is a fundamental feature of human activity, extending beyond language to broader cognitive modeling.

Summary updated

BibTeX
@inproceedings{kayser1981variable,
  author = {Daniel Kayser and Daniel Coulon},
  title = {{Variable-Depth Natural Language Understanding}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1981},
}
Figure