Abstract
Large Language Models (LLMs) often contain latent domain knowledge from pretraining that remains underutilized in the "long tail" of their probability distributions. We present Select2Know (S2K), a framework that intelligently fuses internal parametric knowledge with external documents at the token level. By focusing on "unmastered" knowledge and employing reasoning-enhanced training, S2K enables models to match the performance of massive domain-specific models while requiring up to 1,000x less training data.
Introduction
Adapting LLMs to specialized domains like medicine, law, or finance typically involves a trade-off between Retrieval-Augmented Generation (RAG) and Continued Pretraining (CPT). RAG is flexible but prone to hallucinations from noisy retrieval, while CPT is computationally expensive, requiring billions of tokens. S2K addresses this by identifying that LLMs often possess "unmastered" or partially internalized knowledge. The framework treats domain adaptation as a two-step process: mastering core concepts through internal-external selection and then applying them via complex reasoning.
Methodology
The S2K framework consists of three primary stages designed for cost-effective post-training.
- Internal-External Knowledge Self-Selection: The model performs parallel inferences—one with an external document and one without. By comparing token-level probabilities, the model selects the "internal" token if it is more confident on its own; otherwise, it adopts the "external" token. This creates a high-quality "fusion" dataset.
- Selective Supervised Fine-Tuning (Selective SFT): Instead of uniform weighting, Selective SFT uses token-level entropy to identify unfamiliar domains. Higher loss weights are assigned to these "unmastered" tokens, focusing training on new information.
- Reasoning-Enhanced Training: To improve problem-solving, S2K generates deductive, inductive, and case-based reasoning data. It then applies Group Relative Policy Optimization (GRPO) to refine the model's reasoning capabilities.
Experimental Evaluation
S2K was evaluated across three high-stakes domains: Medicine (MedQA), Law (JEC-QA), and Finance (FinanceIQ). Training data was synthesized from raw domain corpora rather than human-annotated pairs.
Key Results
Experimental results demonstrate that S2K significantly outperforms standard RAG and few-shot prompting. Notably, S2K matches or exceeds the performance of specialized models like BioMistral and Saul-7B. Most importantly, S2K achieves this with a massive reduction in data requirements. For instance, in the medical domain, S2K reached competitive performance using only 0.04 billion tokens, compared to the 3 billion tokens required by BioMistral—representing a 2-to-3 order of magnitude reduction in training data and associated compute costs.
Discussion and Conclusion
S2K proves that "more data" is not always the answer; rather, focusing on the model's specific knowledge gaps is more efficient. While currently focused on static domain knowledge, future work will explore integrating RAG for real-time updates and scaling the framework to larger architectures.
Summary updated