Abstract
This 1986 paper from Yale University argues that natural language parsing and logical inference should not be separate modules, but rather a single, uniform process of searching and modifying a richly indexed memory. By treating "understanding" as the recognition of existing memory structures—and learning as the resolution of "failures" to find a perfect match—the authors present a system that can automatically organize new knowledge as it reads.
The Problem: The "Build and Store" Bottleneck
In the mid-80s, most natural language processing (NLP) systems followed a "Build and Store" model: first, they parsed a sentence into a standalone meaning representation, and then they tried to figure out how that representation fit into the system's long-term memory.
The authors argue this approach is flawed because it prevents the system from using its existing knowledge during the parsing process. Furthermore, it makes learning difficult. If a system doesn't understand something, it shouldn't just fail; it should use that failure as a signal to reorganize its memory or infer new relationships. The goal of the Yale Economics Learning Project was to create a system that could transition from a "novice" to an "expert" by reading economic texts and gradually refining its internal model of how the world works.
Methods: Direct Memory Access Parsing (DMAP)
The authors introduce an architecture based on Direct Memory Access Parsing (DMAP). Key components include:
- Memory Organization Packets (MOPs): Frame-like structures that store domain knowledge (e.g., "Interest rates rise") organized in abstraction hierarchies (class-subclass) and part-whole relationships.
- Concept Sequences: Linguistic templates attached directly to memory structures. For example, the concept of a "communication event" might have a template like
{actor says mobject}. - Marker Passing: The system uses two types of markers to navigate memory: Activation Markers (placed on concepts mentioned in text) and Prediction Markers (representing expectations).
The system operates by a "Recognize and Modify" cycle. Instead of building new structures, it searches for the most specific existing structure in memory that matches the input. If the match is imperfect—for instance, if the text mentions "John Doe" saying something usually attributed to "Milton Friedman"—it triggers a Specialization Failure.
Learning via Failure and Reconciliation
Learning in this framework is not a separate "mode" but a natural byproduct of the system failing to find a perfect match in its memory. These failures (like an unexpected actor or a missing causal link) force the system to create new, more specialized memory structures.
When a failure occurs, the system uses Reconciliation Strategies—high-level inference rules—to fix it. For example, if two different people say the same thing, the system might invoke a CREATE-CAMP strategy, inferring that both people belong to the same "economic camp" (e.g., Monetarists).
Key Insights and Results
- Parsing is Memory Search: Understanding a sentence is essentially finding a specific "file" in a massive, cross-referenced filing cabinet.
- Uniformity of Representation: By using the same format for domain knowledge, linguistic knowledge, and inference rules, the system applies a single search algorithm to all tasks.
- Automatic Re-indexing: When the system encounters a "failure," it creates new structures (like "Economic Camps") that better organize its memory for future use.
- Causal Chaining: The system can bridge gaps in text. If it knows "A causes B" and "B causes C," and it reads "A causes C," it can use a reconciliation strategy to verify the underlying causal chain even if the text is missing the middle step.
Limitations and Future Work
While the DMAP approach avoids some combinatorial explosions, the authors acknowledge several limitations:
- Manual Knowledge Engineering: Initial MOPs and reconciliation strategies must be manually defined. The system learns within this framework but doesn't build it from scratch.
- Scaling: Managing a massive, real-world knowledge base would require more sophisticated "focus of attention" mechanisms.
- Linguistic Learning: A major future goal is the ability for the system to learn new "concept sequences" (linguistic patterns) automatically as it encounters new ways of expressing ideas.
Summary updated