Abstract
Large language models (LLMs) often suffer from reasoning inconsistency, failing in situations unfamiliar to their training data even when the underlying logic is identical to common cases they solve successfully. This familiarity bias suggests that models memorize patterns rather than internalizing abstract reasoning processes. We propose Self-supervised Analogical Learning (SAL), a framework that encourages models to understand high-level symbolic reasoning. SAL mimics the human analogy process by training models to explicitly transfer high-quality symbolic solutions (Python programs) from known cases to rare cases where they typically fail. Our results demonstrate that SAL-trained models outperform base LLMs on benchmarks like StrategyQA, GSM8K, and HotpotQA by 2% to 20%, while exhibiting superior generalizability and controllability.
Introduction: The Familiarity Bias in LLMs
Despite their impressive capabilities, LLMs frequently rely on "familiarity" rather than true logic. For instance, a model might correctly deduce a fact about a famous historical figure but fail the same logical test when applied to a less common entity. This reasoning inconsistency occurs because models tend to memorize specific token sequences rather than the "logical skeleton" of a problem. The SAL framework aims to bridge this gap by forcing models to extract and apply abstract reasoning paths, allowing them to transfer knowledge from common scenarios to the long tail of rare events.
The SAL Framework
SAL utilizes two primary self-supervision extraction pipelines to generate high-quality training data without human intervention.
Conceptualization
In this pipeline, the model abstracts an original question into a template. For example, "Does a mongoose have camouflage for the desert?" becomes "Does Animal X have camouflage for Environment Y?". The model then generates similar but easier questions based on this template. For questions the model can solve with high confidence, it generates a symbolic solution in the form of a Python program. These programs serve as the training targets for the original, more complex questions.
Simplification
For mathematical reasoning, SAL employs an iterative decomposition strategy. Complex problems are broken down into simpler "single-hop" sub-questions. By solving these sub-routines individually, the model reduces cognitive overload. The resulting programmatic steps are then aggregated to form a robust supervision signal, teaching the model to handle multi-step logic systematically.
Experimental Setup and Datasets
We evaluated SAL across a diverse set of reasoning benchmarks:
- StrategyQA: Common-sense reasoning requiring multi-step logic.
- GSM8K: Grade-school math word problems.
- HotpotQA: Multi-hop question answering.
- ARC (Challenge): Difficult science-based questions.
- CommonsenseQA: General knowledge reasoning.
To test generalization, we created "seen" splits for StrategyQA and GSM8K, specifically measuring how well the model transfers logic from training examples to unfamiliar test cases.
Key Results and Insights
SAL-trained models, primarily based on Mistral-7B, showed substantial improvements over standard Chain-of-Thought (CoT) prompting.
As shown in the results, the SAL (all) + RAG configuration achieved significant gains, such as reaching 62.9% on StrategyQA compared to the 58.8% baseline. Notable insights include:
- Programs as Rigid Templates: Using Python code as an intermediate reasoning step prevents the model from "drifting" during complex multi-step tasks.
- Superior Generalization: On conceptual reasoning tasks, SAL-trained Mistral-7B models outperformed significantly larger models like Mixtral-8x7B and Llama-70B.
- Enhanced Controllability: Because the model outputs executable code, users can inspect the logic, identify factual errors in the
ask_llmcalls, and manually correct reasoning paths if necessary.
Conclusion and Future Work
SAL demonstrates that teaching models the process of reasoning via analogy is more effective than simply scaling parameters. While currently optimized for binary and mathematical questions, future work will focus on expanding the conceptualization pipeline to open-ended questions and scaling the self-supervision to millions of examples. By separating logic from factual retrieval, SAL provides a path toward more consistent and interpretable AI reasoning.