Abstract
Large Language Models (LLMs) typically store information across billions of parameters in a distributed, "black box" fashion, making it nearly impossible to trace specific facts back to their source documents. This lack of traceability contributes to hallucinations, privacy concerns, and poor interpretability. Researchers from KAIST and INEEJI propose a novel architectural approach and a specialized loss function designed to force LLMs to store document-specific information in predictable, indexed locations. By utilizing document-specific vector representations (DocReps) and a "document guidance loss," the authors demonstrate that it is possible to disentangle memories and improve recall fidelity.
Introduction
When an LLM internalizes a document during training, the resulting knowledge is often smeared across "polysemantic" neurons—individual parameters that respond to multiple, unrelated concepts. This entanglement makes it difficult to identify which parameters are responsible for specific pieces of information. If a model memorizes sensitive or copyrighted data, there is no straightforward way to "delete" that knowledge without retraining the entire model. Furthermore, this shared memory structure often leads to hallucinations, where the model mixes facts from different sources.
To address these issues, this research moves away from post-hoc interpretability—which attempts to explain model behavior after training—toward design-based traceability. The goal is to structure the model so that it organizes its own memory during the learning process.
Methodology
The authors introduce two primary innovations to the standard Transformer architecture: a document-wise memory structure and a guidance-based loss function.
1. Document-Wise Memory Architecture
In standard Transformers, Multi-Layer Perceptrons (MLPs) act as key-value memories. The proposed architecture modifies this by introducing a DocRep (Document Representation) vector. A dedicated Key Generator maps this DocRep to a set of document-specific memory entries ($Key_{Doc}$).
During the forward pass, the model performs an element-wise multiplication (Hadamard product) between the standard token-based keys and these document-based keys. In effect, the DocRep acts as a soft mask, selectively "turning on" specific neural pathways for specific documents. This ensures that the model utilizes designated parameters for specific content.
2. Document Guidance Loss
To ensure that information from Document A is strictly confined to its assigned memory slot, the researchers adapted "classifier-free guidance," a technique often used in diffusion models. This involves a dual-action loss function:
- The Positive Guidance ("Push"): The loss function increases the likelihood of generating the document's text when the correct DocRep is provided.
- The Negative Guidance ("Pull"): Crucially, the function decreases the likelihood of generating that same text when an incorrect or "negative" DocRep (such as a zero vector or a random vector) is used.
This "push-pull" mechanism forces the model to disentangle information, preventing the contents of one document from leaking into the memory slots of another.
Experimental Setup
The researchers evaluated their approach using the Pythia 1B model trained on subsets of the Wikitext-103-v1 dataset. They tested the model's ability to memorize and recall 10, 20, and 50 distinct documents. Performance was measured using standard ROUGE scores for recall and a new metric, IV-ROUGE (Inverse ROUGE), which quantifies cross-document contamination (leakage).
Key Insights
The Power of Linearity
A significant finding was that a simple linear mapping from DocReps to memory keys outperformed complex, non-linear MLPs. The authors suggest this is due to "Lipschitz continuity," where smooth transitions in the document representation space lead to stable changes in the memory space, facilitating the convergence of the guidance loss.
Activation Functions and Inductive Bias
The choice of activation function for the keys proved critical. ReLU and Tanh were found to be superior to Sigmoid. ReLU, in particular, functions as a "harder" switch, which helps create distinct, non-overlapping memory entries, further aiding the isolation of document-specific facts.
Disentanglement vs. Memorization Speed
While standard shared-memory models are slightly faster at initial memorization, they are prone to "knowledge mixing." The proposed guidance method, while requiring more training iterations to converge, results in significantly higher "fact fidelity." In qualitative tests, the guidance-based model correctly recalled document-specific details (e.g., matching a song to the correct artist) where the baseline model conflated facts.
Results
The proposed method achieved ROUGE-1 precision scores of approximately 0.80 when using Tanh or ReLU guidance, compared to only 0.61 for standard MLP baselines. Furthermore, the IV-ROUGE scores were significantly lower, confirming that the model successfully isolated document contents and reduced cross-contamination.
Limitations and Future Work
While promising, the study identifies several areas for further research:
- Scalability: The current experiments were limited to 50 documents. Scaling to the billions of documents found in massive corpora may require a hierarchical document structure.
- Generalization: The research focused on memorization and recall; it is yet to be determined how these architectural changes affect zero-shot reasoning or general linguistic capabilities.
- Semantic Purity: Occasional "false positives" occur where unrelated content seeps into memory entries, suggesting a need for further refinement of the DocRep generation process.
Conclusion
By integrating document-specific guidance directly into the model architecture, this research provides a path toward more transparent and reliable Large Language Models. Moving from "black box" parameters to indexed, traceable memory slots could significantly improve the safety, privacy, and accuracy of future AI systems.
Summary updated