TL;DR
This paper introduces a "shared organizational cortex," a four-layer knowledge architecture designed to persist and maintain the interpretations organizations derive from evidence. By combining symbolic truth-maintenance with geometric dimensions, the system ensures that judgments remain valid, revisable, and reusable even as the underlying data changes.
What Problem Does This Paper Address?
Organizations are drowning in data—conversations, documents, and reports—but they act on interpretations: the judgments and "theories of change" assembled from multiple observations. Current AI systems, including standard RAG (Retrieval-Augmented Generation), knowledge graphs, and agent memories, are excellent at keeping records but terrible at preserving the reasoning behind them.
When a conclusion travels without its supporting evidence or the assumptions that might invalidate it, we encounter "uncertainty absorption." If the underlying evidence changes (e.g., a partner withdraws from a project), traditional systems often fail to update the dependent judgments. The authors aim to build a system that treats interpretations as first-class, maintainable objects rather than transient outputs of a single LLM prompt.
Methods
The authors propose a four-layer architecture to bridge the gap between raw data and actionable judgment:
- Signals: Atomic, attributable propositions extracted from source material ("Entries").
- Domains: Persistent semantic boundaries (e.g., "Climate Change" or "Kenya") that scope signals and queries.
- Claims: The core of the architecture. These are persisted interpretations justified by one or more Signals or prior Claims. They include explicit preconditions and "standing."
- Dimensions: Named, versioned axes (e.g., "funding availability" or "implementation feasibility") that provide a geometric structure for positioning and comparing Claims.
A key technical contribution is the Truth Maintenance System (TMS). Adapted from classic AI research (Doyle, 1979), this dependency network manages the "standing" of a Claim. A Claim can be proposed, accepted, contested, retracted, or superseded. If a supporting Signal is withdrawn, the TMS automatically propagates that loss of support, retracting the dependent Claim.
Furthermore, the system uses Geometric Knowledge Representation. By projecting Claims onto Dimensions, the system can detect contradictions and rank search results more effectively than simple vector similarity.
Datasets
The researchers evaluated their architecture using:
- Pilot Kit: 121 Signals derived from 49 Python Enhancement Proposals (PEPs) and 56 Signals from a subset of the Enron email corpus.
- Retrieval Evaluation: A registered experiment involving 272 pooled held-out queries across three distinct workspaces.
- Annotator Calibration: A five-item calibration set used to measure agreement between human annotators on relevance and dimension positioning.
Key Insights and Ideas
- Interpretations as Materialized Views: Rather than re-deriving a judgment every time a user asks a question, the system stores the judgment (the Claim) and its justification graph. This allows for "organizational instinct"—the ability to reuse prior justified reasoning.
- Symbolic + Geometric Hybrid: The system uses symbolic logic (justification graphs) to handle truth and provenance, while using geometry (Dimensions) to handle graded qualities and similarity.
- Contradiction as "Contested" Standing: Instead of a binary true/false, the system handles conflict by marking Claims as "contested." This preserves the disagreement for the user to inspect rather than simply deleting data.
- Atomic Commits for Knowledge: Changes to the cortex (like splitting a Domain or superseding a Signal) are processed in atomic batches, ensuring that users always see a consistent state of the organization's "mind."
Key Results
The architecture showed significant improvements in retrieval performance:
- Dimension Gain: Adding the Dimension-aware reranking term improved nDCG@10 by 0.028 (p = 0.002).
- System-Level Performance: The full architecture outperformed a "Signals-only" baseline by 0.081 nDCG@10 and beat a graph-augmented RAG baseline by 0.053 (both p < 0.001).
- Reliability: In pilot tests, human annotators showed high agreement on Dimension authoring (Spearman correlations up to 0.94), suggesting that these geometric axes are a stable way to represent organizational knowledge.
Limitations and Future Work
- Maintenance Overhead: The authors acknowledge that truth maintenance imposes a curation burden that may not be suitable for every workload.
- Recall Challenges: Both the "Extractor" (finding signals) and "Contradiction Check" (finding conflicts) are fallible; improving their recall is a primary area for future work.
- Governance: The paper does not deeply explore the social/permission layer—specifically, who within an organization has the authority to "accept" or "reject" a Claim.
- Complex Reasoning: Future iterations could explore more complex conflict chains using formal argumentation semantics.
Summary updated