RECKONING: Reasoning through Dynamic Knowledge Encoding

Zeming Chen, Gail Weiss, Eric Mitchell, Asli Celikyilmaz, Antoine Bosselut

Abstract

Recent studies on transformer-based language models show that they can answer questions by reasoning over knowledge provided as part of the context (i.e., in-context reasoning). However, since the available knowledge is often not filtered for a particular question, in-context reasoning can be sensitive to distractor facts, additional content that is irrelevant to a question but that may be relevant for a different question (i.e., not necessarily random noise). In these situations, the model fails to distinguish the knowledge that is necessary to answer the question, leading to spurious reasoning and degraded performance. This reasoning failure contrasts with the model's apparent ability to distinguish its contextual knowledge from all the knowledge it has memorized during pre-training. Following this observation, we propose teaching the model to reason more robustly by folding the provided contextual knowledge into the model's parameters before presenting it with a question. Our method, RECKONING, is a bi-level learning algorithm that teaches language models to reason by updating their parametric knowledge through back-propagation, allowing them to then answer questions using the updated parameters. During training, the inner loop rapidly adapts a copy of the model weights to encode contextual knowledge into its parameters. In the outer loop, the model learns to use the updated weights to reproduce and answer reasoning questions about the memorized knowledge. Our experiments on two multi-hop reasoning datasets show that RECKONING's performance improves over the in-context reasoning baseline (by up to 4.5%). We also find that compared to in-context reasoning, RECKONING generalizes better to longer reasoning chains unseen during training, is more robust to distractors in the context, and is more computationally efficient when multiple questions are asked about the same knowledge.

Introduction

Most modern Large Language Models (LLMs) perform reasoning through In-Context Reasoning (ICR), where a set of facts is provided in the prompt followed by a question. While effective, ICR is highly sensitive to distractors—irrelevant facts that, while coherent, do not contribute to the specific reasoning path required. This sensitivity creates a paradox: LLMs are easily confused by distractors in their input context, yet they are remarkably robust to distractors within their own parametric knowledge (the billions of facts memorized during pre-training).

We introduce RECKONING (REasoning through dynamiC KnOwledge eNcodING), a bi-level optimization framework designed to make contextual knowledge behave more like internal parametric knowledge. By "folding" new information directly into the model's weights via fast gradient updates, we enable more robust and efficient multi-hop reasoning.

Methodology

RECKONING employs a bi-level learning algorithm inspired by meta-learning. The process is divided into two distinct loops:

  1. The Inner Loop (Encoding): Given a set of new facts, the model performs a few steps of gradient descent to encode these facts into a temporary copy of its weights. This process effectively "memorizes" the context into the model's parameters.
  2. The Outer Loop (Reasoning): The model's meta-parameters are optimized such that, after the inner-loop adaptation, the updated model can successfully answer reasoning questions and reproduce the original facts. This dual objective—Reasoning and Knowledge Recovery—is crucial for maintaining information integrity during the weight update process.
The RECKONING bi-level learning architecture, showing the inner-loop knowledge encoding and the outer-loop reasoning optimization.

Experimental Evaluation

We evaluated RECKONING on several multi-hop reasoning benchmarks, including ProofWriter (logical deductions), CLUTRR-SG (kinship relations), and FOLIO (expert-annotated first-order logic).

Key Results

  • Performance Gains: RECKONING outperformed fine-tuned in-context reasoning (FT-ICR) baselines by up to 4.5% across benchmarks. On the real-world FOLIO dataset, a RECKONING-enhanced GPT-2 base (1.5B) outperformed zero-shot GPT-3.5 and ChatGPT (54.9% vs 45.1%).
  • Distractor Robustness: RECKONING exhibits significantly higher resilience to irrelevant information. On ProofWriter with full distractors, RECKONING achieved 82.5% accuracy compared to 70.9% for the ICR baseline.
  • Systematic Generalization: The method generalizes better to unseen reasoning depths. Models trained on 2-hop reasoning maintained higher accuracy when tested on 6-hop or 8-hop problems compared to standard methods.
  • Computational Efficiency: While the initial encoding step is computationally intensive, RECKONING becomes more efficient when multiple questions are asked about the same context. In tests with 18 questions per context, it achieved a ~3x speedup over ICR.
Label accuracy comparison between RECKONING and FT-ICR as the number of distractor facts increases across different reasoning hop counts.

Conclusion

RECKONING demonstrates that internalizing contextual knowledge into model weights provides a more robust foundation for reasoning than traditional in-context methods. By leveraging the inherent robustness of parametric memory, RECKONING mitigates the impact of distractors and improves generalization to complex reasoning chains. Future work will focus on scaling this approach to larger models and improving the stability of the bi-level optimization process.

Summary updated

Abstract

Recent studies on transformer-based language models show that they can answer questions by reasoning over knowledge provided as part of the context (i.e., in-context reasoning). However, since the available knowledge is often not filtered for a particular question, in-context reasoning can be sensitive to distractor facts, additional content that is irrelevant to a question but that may be relevant for a different question (i.e., not necessarily random noise). In these situations, the model fails to distinguish the knowledge that is necessary to answer the question, leading to spurious reasoning and degraded performance. This reasoning failure contrasts with the model's apparent ability to distinguish its contextual knowledge from all the knowledge it has memorized during pre-training. Following this observation, we propose teaching the model to reason more robustly by folding the provided contextual knowledge into the model's parameters before presenting it with a question. Our method, RECKONING, is a bi-level learning algorithm that teaches language models to reason by updating their parametric knowledge through back-propagation, allowing them to then answer questions using the updated parameters. During training, the inner loop rapidly adapts a copy of the model weights to encode contextual knowledge into its parameters. In the outer loop, the model learns to use the updated weights to reproduce and answer reasoning questions about the memorized knowledge. Our experiments on two multi-hop reasoning datasets show that RECKONING's performance improves over the in-context reasoning baseline (by up to 4.5%). We also find that compared to in-context reasoning, RECKONING generalizes better to longer reasoning chains unseen during training, is more robust to distractors in the context, and is more computationally efficient when multiple questions are asked about the same knowledge.
BibTeX
@inproceedings{chen2023reckoning,
  author = {Zeming Chen and Gail Weiss and Eric Mitchell and Asli Celikyilmaz and Antoine Bosselut},
  title = {{RECKONING: Reasoning through Dynamic Knowledge Encoding}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=dUAcAtCuKk},
}
Figure