Transformers as Soft Reasoners over Language

Peter Clark, Oyvind Tafjord, Kyle Richardson

Abstract

Beginning with McCarthy's Advice Taker (1959), AI has pursued the goal of providing a system with explicit, general knowledge and having the system reason over that knowledge. However, expressing the knowledge in a formal (logical or probabilistic) representation has been a major obstacle to this research. This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data. Our models, that we call RuleTakers, provide the first empirical demonstration that this kind of soft reasoning over language is learnable, can achieve high (99%) accuracy, and generalizes to test data requiring substantially deeper chaining than seen during training (95%+ scores). We also demonstrate that the models transfer well to two hand-authored rulebases, and to rulebases paraphrased into more natural language. These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language. This in turn suggests new possibilities for explainability, correctability, and counterfactual reasoning in question-answering.

Introduction: Bypassing the Logic Bottleneck

For decades, the "Holy Grail" of AI has been a system capable of taking a set of facts and rules, reasoning over them, and arriving at logical conclusions. Traditionally, this required symbolic AI: the arduous process of translating natural language into formal logic representations (like Prolog) to be processed by a solver. However, this "semantic parsing" step is notoriously brittle; minor linguistic ambiguities can cause the entire system to fail.

This research explores a provocative alternative: skipping formal logic entirely. Instead of translating text to symbols, can we train a Transformer to emulate the behavior of a logical reasoner directly on natural language? By treating reasoning as a learnable skill, we may gain the benefits of logical rigor without the overhead of formal translation.

Methodology: Soft Theorem Proving

The researchers adopted a strategy called algorithm emulation. First, they generated thousands of "theories" using formal logic. These theories were then translated into simple English sentences using templates (e.g., is(Alan, Blue) becomes "Alan is blue").

Comparison between traditional formal reasoning and the proposed linguistic analog where the model operates directly on natural language.

The resulting model, a fine-tuned RoBERTa-large dubbed a "RuleTaker," was trained on (context, question, answer) triples. The model's task was to predict if a statement was True or False based solely on the provided text. This approach allows the model to function as a "soft theorem prover," handling minor linguistic variations that would typically crash a symbolic solver.

Datasets and Experimental Setup

The study utilized several distinct datasets to test different facets of reasoning:

  • Synthetic Depth Datasets (D=0 to D≤5): These contain rules requiring varying "hops" of reasoning. $D=0$ is a simple lookup, while $D=5$ requires chaining five rules together.
  • Birds & Electricity: Two hand-authored rulebases used for zero-shot testing. "Birds" involves classic logic puzzles (e.g., ostriches and flight), while "Electricity" describes circuit behavior.
  • ParaRules: A dataset of 40,000 examples where synthetic rules were rephrased by crowdworkers into natural, creative English to test linguistic robustness.

Key Results and Insights

Near-Perfect Accuracy and Generalization

RoBERTa achieved 99% accuracy on the synthetic test sets. Most impressively, the model demonstrated strong generalization: when trained on reasoning depths only up to 3, it still solved $D=5$ problems with 95%+ accuracy. This suggests the model is learning the underlying procedure of deduction rather than just memorizing patterns.

Explainability via Perturbation

To verify that the model was truly reasoning, the researchers performed a perturbation analysis by removing "critical sentences" (facts necessary for a proof).

Table showing the results of perturbation analysis: when a critical fact is removed, the model's prediction correctly flips from True to False in the vast majority of cases.

As shown in the analysis, removing a necessary fact flipped the model's prediction from True to False 81% of the time (16,654 correct flips). This indicates that the model "understands" which facts are essential to its conclusion.

Robustness and Transfer

The models transferred remarkably well to the hand-authored "Birds" and "Electricity" problems, achieving 90%+ accuracy without specific training on those vocabularies. While zero-shot performance on the varied ParaRules was initially lower (66.6%), minimal fine-tuning brought it to 98.8%, proving the model can map natural phrasing to its internal reasoning logic.

Conclusion and Future Work

These findings suggest a new role for Transformers as soft reasoners over explicit theories in language. While the current work is limited to conjunctive rules and small theories (under 20 facts), it opens new possibilities for explainable and correctable question-answering. Future research will need to address more complex logic—such as disjunctions and universal quantifiers—and scale these methods to massive, potentially contradictory knowledge bases like Wikipedia.

Summary updated

Abstract

Beginning with McCarthy's Advice Taker (1959), AI has pursued the goal of providing a system with explicit, general knowledge and having the system reason over that knowledge. However, expressing the knowledge in a formal (logical or probabilistic) representation has been a major obstacle to this research. This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data. Our models, that we call RuleTakers, provide the first empirical demonstration that this kind of soft reasoning over language is learnable, can achieve high (99%) accuracy, and generalizes to test data requiring substantially deeper chaining than seen during training (95%+ scores). We also demonstrate that the models transfer well to two hand-authored rulebases, and to rulebases paraphrased into more natural language. These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language. This in turn suggests new possibilities for explainability, correctability, and counterfactual reasoning in question-answering.
BibTeX
@inproceedings{clark2020transformers,
  author = {Peter Clark and Oyvind Tafjord and Kyle Richardson},
  title = {{Transformers as Soft Reasoners over Language}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  url = {https://openreview.net/forum?id=ilJmO_Jitg},
}
Figure