Thrill-K Architecture: Towards a Solution to the Problem of Knowledge Based Understanding

Gadi Singer, Joscha Bach, Tetiana Grinberg, Nagib Hakim, Phillip Ryan Howard, Vasudev Lal, Zev Rivlin

TL;DR

The Thrill-K architecture is a hybrid neuro-symbolic blueprint designed to move AI from simple pattern recognition to "Cognitive AI" capable of human-like understanding. By partitioning knowledge into three distinct layers—instantaneous (neural), standby (structured knowledge base), and external (retrieved)—it aims to solve the sustainability crisis of massive language models while adding much-needed reasoning, explainability, and common sense.

What Problem Does This Paper Address?

Current Deep Learning (DL) is stuck in what psychologists call "System 1" thinking: it is fast, intuitive, and excellent at correlation, but it lacks the "System 2" capabilities of logic, planning, and causal reasoning.

The authors identify three critical walls that current AI is hitting:

  1. Efficiency: Computational requirements for state-of-the-art models are doubling every 3.5 months, a trend that is physically and economically unsustainable.
  2. Robustness: Models struggle to adapt to "unknown unknowns" or contexts outside their training data.
  3. Cognition: DL lacks the mechanisms for abstraction, context, and explainability—the "why" behind the "what."

The paper argues that simply scaling up neural networks won't bridge this gap; instead, we need a fundamental architectural shift that integrates structured, symbolic knowledge with neural learning.

Methods

The authors propose the Thrill-K Architecture (a contraction of "Three Levels of Knowledge"), which categorizes AI systems based on how they handle information:

  • Class 1 (Fully Encapsulated): Standard DL models (like GPT-3) where all knowledge is "baked into" the weights.
  • Class 2 (Semi-Structured Adjacent): Retrieval-augmented systems that look up external text (like Wikipedia) but lack deep structural understanding.
  • Class 3 (Deeply Structured Knowledge): The goal of Thrill-K. These systems use an explicit, multi-dimensional knowledge base (KB) that interacts closely with a neural network.

The Three Levels of Knowledge (3LK): 1. Instantaneous Knowledge: Encoded in the neural network's parametric memory for rapid, "reflexive" responses. 2. Standby Knowledge: Resides in an adjacent Deep Knowledge Base (often a Knowledge Graph). This contains structured facts, causal models, and "scripts" for how the world works. 3. Retrieved External Knowledge: Information pulled from massive, unstructured digital repositories when the internal system lacks specific data.

A key component is the Reasoned Extraction block, which acts as a mediator, allowing the neural network to query the structured knowledge base dynamically during inference.

Datasets

As this is a conceptual and architectural position paper, the authors do not perform new empirical benchmarks or introduce specific training datasets. They reference existing curated systems like the OpenCyc ontology, AMR (Abstract Meaning Representation) named entity types, and Wikipedia as examples of the types of information sources that would populate a Thrill-K system.

Key Insights and Ideas

  • Knowledge Stratification: Not all knowledge belongs in the neural weights. By moving "standby" knowledge to a structured KB, we can reduce the number of parameters in the neural model by orders of magnitude without losing accuracy.
  • The Six Dimensions of Knowledge: The authors argue that "understanding" requires more than just facts. A true Cognitive AI needs:
  • Descriptive knowledge (facts/hierarchies).
  • Dynamic models (how the world changes/physics).
  • Stories and scripts (social contexts/narratives).
  • Context and source attribution (who said what and when).
  • Values and priorities (ethics and goals).
  • Concept references (unifying different representations of the same thing).
  • Dynamic Updating: Unlike standard LLMs, which require expensive retraining to learn new facts, a Thrill-K system can be updated simply by editing its Knowledge Base.

Key Results

The paper provides a theoretical framework rather than quantitative experimental results. However, it outlines the qualitative advantages of the Thrill-K approach:

  • Explainability: Because the system queries an explicit knowledge base, it can provide a "trace" of the symbolic entities and relations used to reach a conclusion.
  • Sustainability: By offloading memory to a structured KB, the "arms race" of model parameter scaling can be mitigated.
  • System 2 Integration: It provides a concrete blueprint for how to combine gradient-descent-based learning with symbolic reasoning.

Limitations and Future Work

The authors acknowledge that building a Thrill-K system is significantly more complex than training an end-to-end neural network. Major hurdles include:

  • Knowledge Engineering: Creating and maintaining a "Deep Knowledge Base" that captures the richness of human experience is a massive undertaking.
  • Hybrid Learning: We currently lack standardized algorithms that can seamlessly integrate statistical gradient descent with symbolic logic and learning.
  • Reasoning Complexity: While the architecture provides a "place" for reasoning to happen (the Reasoned Extraction block), the actual implementation of robust, multi-step symbolic reasoning over large graphs remains a difficult research challenge.

Summary updated

Abstract

While end-to-end learning systems are rapidly gaining capabilities and popularity, the increasing computational demands for deploying such systems, along with a lack of flexibility, adaptability, explainability, reasoning and verification capabilities, require new types of architectures. Here we introduce a classification of hybrid systems which, based on an analysis of human knowledge and intelligence, combines neural learning with various types of knowledge and knowledge sources. We present the Thrill-K architecture as a prototypical solution for integrating instantaneous knowledge, standby knowledge and external knowledge sources in a framework capable of inference, learning and intelligent control.
BibTeX
@inproceedings{singer2022thrill,
  author = {Gadi Singer and Joscha Bach and Tetiana Grinberg and Nagib Hakim and Phillip Ryan Howard and Vasudev Lal and Zev Rivlin},
  title = {{Thrill-K Architecture: Towards a Solution to the Problem of Knowledge Based Understanding}},
  booktitle = {AGI},
  year = {2022},
  url = {https://openreview.net/forum?id=wPT9rUNYOo},
}

Loading related content...

Figure