An On-Line Computational Model of Human Sentence Interpretation

Daniel JurafskyAAAI 1992 · OpenReview

Abstract

This dissertation presents Sal, a computational model of human sentence interpretation that integrates linguistic theory, computer science, and cognitive psychology. Sal utilizes a Construction-Based Interpretative Grammar (CIG) to represent lexical, syntactic, and semantic knowledge uniformly. As an "on-line" interpreter, Sal processes sentences word-by-word, updating partial interpretations through an evidential mechanism that balances top-down and bottom-up information. The model successfully accounts for various psycholinguistic phenomena, including garden-path effects, by employing a coherence-based selection and pruning mechanism.

Introduction

In the early 1990s, the study of language was fragmented across disciplines. Linguists developed complex theories often divorced from computational feasibility, while computer scientists built efficient parsers that lacked cognitive plausibility. Psychologists, meanwhile, studied human processing behavior without the benefit of integrated computational models. Daniel Jurafsky’s 1992 dissertation bridges these gaps with Sal, a model that interprets sentences in real-time, mirroring the incremental nature of human language processing.

The central challenge addressed is on-line interpretation ambiguity. Humans do not wait until the end of a sentence to begin understanding it; we interpret as we hear or read. This creates a massive search space of potential meanings. Sal manages this complexity through a uniform representation of knowledge and a robust pruning algorithm.

Construction-Based Interpretive Grammar (CIG)

The foundation of Sal is CIG, which rejects the traditional distinction between a "dictionary" (lexicon) and "rules" (syntax). Instead, every linguistic unit—from a single word to a complex idiom or a high-level grammatical rule—is represented as a construction.

The 'Create' construction, illustrating the unification of lexical, semantic, and frequency information.

As shown in the "Create" construction above, each entry includes its name, frequency of occurrence (derived from the Brown Corpus), semantic constituents (e.g., Creator and Created), and graphemic constraints. This uniformity allows a single processing mechanism to handle all levels of linguistic integration.

The Sal Interpreter Architecture

The Sal engine operates through a series of stages that move information from long-term memory into active interpretation.

The architecture of the Sal interpreter, showing the flow from long-term grammar store to the access buffer and interpretation store.

The process involves four key functions:

  1. Access: Constructions are pulled from the long-term "Grammar" store into an "Access Buffer" based on evidence from the input string and current context.
  2. Integration: A unification-like process merges the meanings of accessed constructions.
  3. Selection: Interpretations are ranked in the "Interpretation Store" based on their coherence with grammatical knowledge and semantic context.
  4. Pruning: To prevent an exponential explosion of possibilities, Sal uses a "Selection Timing Principle." If an interpretation's score falls below a certain threshold relative to the best candidate, it is pruned from the working store.

Key Insights: Frequency and Human Error

A major contribution of Sal is treating frequency as a first-class citizen. By incorporating data from the Brown Corpus, Sal prioritizes interpretations that are statistically more likely. This provides a computational explanation for "garden-path" sentences, such as "The horse raced past the barn fell."

In this example, the human brain (and Sal) initially interprets "raced" as the main verb because that usage is highly frequent and coherent. By the time the word "fell" is reached, the correct "reduced relative" interpretation has often been pruned because it fell below the selection threshold. This demonstrates how Sal’s architectural constraints mirror human cognitive limitations.

Results and Implementation

Sal was implemented in Common Lisp with a test grammar of 50 constructions. It successfully demonstrated full traces for sentences like "How can I create disk space?" and accounted for 12 established psycholinguistic benchmarks, including parallel lexical access and the immediate use of verb valence information during integration.

Conclusion

Sal represents a significant step toward a unified theory of language processing. By demonstrating that a single, knowledge-intensive mechanism can handle lexical, syntactic, and semantic integration on-line, Jurafsky provided a blueprint for modern NLP systems that prioritize both statistical likelihood and structural coherence. Future work remains in scaling the "constructicon" and modeling how these complex structures are learned from experience.

Summary updated

Abstract

This dissertation presents a model of the human sentence interpretation process, which attempts to meet criteria of adequacy imposed by the different paradigms of sentence interpretation. These include the need to produce a high-level interpretation, to embed a linguistically motivated grammar, and to be compatible with psycholinguistic results on sentence processing. The model includes a theory of grammar called Construction-Based Interpretative Grammar (CIG) and an interpreter which uses the grammar to build an interpretation for single sentences. An implementation ofhte interpreter has been built called Sal. Sal is an on-line interpreter, reading words one at a time and updating a partial interpretation of the sentence after each constituent. This constituent-by-constituent interpretation is more fine-grained and hence more on-line than most previous models. Sal is strongly interactionist in using both bottom-up and top-down knowledge in an evidential manner to access a set of constructions to build interpretations. It uses a coherence-based selection mechanism to choose among these candidate interpretations, and allows temporary limited parallelism to handle local ambiguities. Sal''s architecture is consistent with a large number of psycholinguistic results. The interpreter embodies a number of strong claims about sentence processing. One claim is uniformity, with respect to both representation and process. In the grammar, a single kind of knowledge structure, the grammatical construction, is used to represent lexical, syntactic, idiomatic, and semantic knowledge. CIG thus does not distinguish between the lexicon, the idiom dictionary, the syntactic rule base, and the semantic rule base. Uniformity in processing means that there is no distinction between the lexical analyzer, the parser, and the semantic interpreter. Because these kinds of knowledge are represented uniformly, they can be accessed, integrated, and disambiguated by a single mechanism. A second claim the interpreter embodies is that sentence processing is fundamentally knowledge-intensive and expectation-based. The representation and integration of constructions uses many diverse types of linguistic knowledge. Similarly, the access of constructions is sensitive to top-down and bottom-up, syntactic and semantic knowledge, and the selection of constructions is based on coherence with grammatical knowledge and the interpretation.
BibTeX
@inproceedings{jurafsky1992line,
  author = {Daniel Jurafsky},
  title = {{An On-Line Computational Model of Human Sentence Interpretation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1992},
  url = {https://openreview.net/forum?id=B1-JWeWd-S},
}
Figure