Abstract
Beginning with McCarthy's Advice Taker (1959), AI has pursued the goal of providing a system with explicit, general knowledge and having the system reason over that knowledge. However, expressing the knowledge in a formal (logical or probabilistic) representation has been a major obstacle to this research. This paper investigates a modern approach to this problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to reason (or emulate reasoning) over these sentences using synthetically generated data. Our models, that we call RuleTakers, provide the first empirical demonstration that this kind of soft reasoning over language is learnable, can achieve high (99%) accuracy, and generalizes to test data requiring substantially deeper chaining than seen during training (95%+ scores). We also demonstrate that the models transfer well to two hand-authored rulebases, and to rulebases paraphrased into more natural language. These findings are significant as it suggests a new role for transformers, namely as limited "soft theorem provers" operating over explicit theories in language. This in turn suggests new possibilities for explainability, correctability, and counterfactual reasoning in question-answering.
Introduction: Bypassing the Logic Bottleneck
For decades, the "Holy Grail" of AI has been a system capable of taking a set of facts and rules, reasoning over them, and arriving at logical conclusions. Traditionally, this required symbolic AI: the arduous process of translating natural language into formal logic representations (like Prolog) to be processed by a solver. However, this "semantic parsing" step is notoriously brittle; minor linguistic ambiguities can cause the entire system to fail.
This research explores a provocative alternative: skipping formal logic entirely. Instead of translating text to symbols, can we train a Transformer to emulate the behavior of a logical reasoner directly on natural language? By treating reasoning as a learnable skill, we may gain the benefits of logical rigor without the overhead of formal translation.
Methodology: Soft Theorem Proving
The researchers adopted a strategy called algorithm emulation. First, they generated thousands of "theories" using formal logic. These theories were then translated into simple English sentences using templates (e.g., is(Alan, Blue) becomes "Alan is blue").
The resulting model, a fine-tuned RoBERTa-large dubbed a "RuleTaker," was trained on (context, question, answer) triples. The model's task was to predict if a statement was True or False based solely on the provided text. This approach allows the model to function as a "soft theorem prover," handling minor linguistic variations that would typically crash a symbolic solver.
Datasets and Experimental Setup
The study utilized several distinct datasets to test different facets of reasoning:
- Synthetic Depth Datasets (D=0 to D≤5): These contain rules requiring varying "hops" of reasoning. $D=0$ is a simple lookup, while $D=5$ requires chaining five rules together.
- Birds & Electricity: Two hand-authored rulebases used for zero-shot testing. "Birds" involves classic logic puzzles (e.g., ostriches and flight), while "Electricity" describes circuit behavior.
- ParaRules: A dataset of 40,000 examples where synthetic rules were rephrased by crowdworkers into natural, creative English to test linguistic robustness.
Key Results and Insights
Near-Perfect Accuracy and Generalization
RoBERTa achieved 99% accuracy on the synthetic test sets. Most impressively, the model demonstrated strong generalization: when trained on reasoning depths only up to 3, it still solved $D=5$ problems with 95%+ accuracy. This suggests the model is learning the underlying procedure of deduction rather than just memorizing patterns.
Explainability via Perturbation
To verify that the model was truly reasoning, the researchers performed a perturbation analysis by removing "critical sentences" (facts necessary for a proof).
As shown in the analysis, removing a necessary fact flipped the model's prediction from True to False 81% of the time (16,654 correct flips). This indicates that the model "understands" which facts are essential to its conclusion.
Robustness and Transfer
The models transferred remarkably well to the hand-authored "Birds" and "Electricity" problems, achieving 90%+ accuracy without specific training on those vocabularies. While zero-shot performance on the varied ParaRules was initially lower (66.6%), minimal fine-tuning brought it to 98.8%, proving the model can map natural phrasing to its internal reasoning logic.
Conclusion and Future Work
These findings suggest a new role for Transformers as soft reasoners over explicit theories in language. While the current work is limited to conjunctive rules and small theories (under 20 facts), it opens new possibilities for explainable and correctable question-answering. Future research will need to address more complex logic—such as disjunctions and universal quantifiers—and scale these methods to massive, potentially contradictory knowledge bases like Wikipedia.
Summary updated