Reasoning Supervision and Capability Retention in LoRA Domain Adaptation

Saleh Ashkboos, Dhruv Desai, Lavinia GhitaarXiv 2026 · arXiv

This study from researchers at NVIDIA explores a critical tension in modern AI: how do we make a model an expert in a specific field (like finance) without it losing the general "intelligence" that made it useful in the first place? By investigating Low-Rank Adaptation (LoRA) fine-tuning on the Qwen3-8B model, the authors provide a systematic look at whether supervising a model with complex reasoning traces—essentially "Chain-of-Thought" (CoT) distillation—actually helps or hinders performance.

TL;DR

While distilling reasoning traces from a larger teacher model can slightly boost general reasoning benchmarks (like MMLU-Pro), simple "response-only" fine-tuning is surprisingly superior for domain-specific classification accuracy and calibration. Adding complex reasoning traces can actually degrade a model's certainty and classification performance due to the shift in context during training.

What Problem Does This Paper Address?

When we fine-tune a Foundation Model for a specialized domain—say, identifying event types in financial news—we often face two problems:

  1. Catastrophic Forgetting: The model gets better at the new task but worse at general reasoning, instruction following, or scientific knowledge.
  2. Supervision Sparsity: A simple label (e.g., "Category A") provides very little signal. Distilling "reasoning traces" from a larger teacher model (like Qwen3-32B) is a popular way to provide more supervision per example, but it’s unclear if this extra "noise" (hedging, repetition, teacher-specific prose) helps a student model learn a specialized decision boundary.

Methods

The authors employed a rigorous, four-stage experimental protocol to ensure that differences in results were due to the training regimes themselves, not just lucky hyperparameter choices.

Figure 1: Overview of the staged protocol. The teacher (Qwen3-32B) labels 200K FinHeadlineMix headlines and generates a reasoning trace for each. We then fine-tune the student (Qwen3-8B) with LoRA in four stages: (1) response-only LoRA, trained on the label alone over a hyperparameter grid, (2) single-adapter reasoning, trained on the trace and the label with the blended loss of Eq. (1), (3) dual-adapter composite LoRA, which splits the rank budget between a trace adapter and a label adapter, and (4) summarized traces, which repeats Stages 2 and 3 with teacher-generated summaries at target lengths of 25%, 50%, and 75%. In every stage, each run uses three seeds and is evaluated on the in-domain task and the retention benchmarks. A run is carried to the next stage when it outperforms the frozen model on every task. If none does, we fall back to the run with the highest average retention score among those that beat the frozen model on the in-domain tasks.
  • Stage 1: Response-only LoRA. The baseline. The model is trained only on the final labels.
  • Stage 2: Single-adapter Reasoning. The model is trained on a "blended loss" that combines the loss from the reasoning trace and the final label, controlled by a weight parameter $\lambda$.
  • Stage 3: Dual-adapter Composite. The rank budget (total $r=32$) is split between two separate adapters—one for the trace and one for the label—which are merged at inference time.
  • Stage 4: Summarized Traces. To reduce the computational cost and potential noise of long traces, the teacher model was asked to summarize its own reasoning before it was used to train the student.

Datasets

  • Training/In-domain: FinHeadlineMix, a dataset of 200,000 financial news headlines classified into 35 event categories.
  • Retention Benchmarks: To measure general capability retention, the authors used MMLU-Pro (general knowledge), GPQA-Diamond (graduate-level science), and IFEval (instruction following).

Key Insights and Ideas

  • The "Effective Learning Rate" is King: The authors found that the most decisive factor in LoRA success wasn't just the learning rate (LR) or the rank ($r$), but the ratio $(\alpha/r \cdot LR)$.
  • Context Matters More Than Signal: A surprising finding was that even when the trace loss was weighted at zero ($\lambda = 1.0$), models trained with traces performed worse than response-only models. Why? Because the presence of the trace changes the context in which the label is learned, shifting the model's internal decision boundaries.
  • Calibration vs. Reasoning: There is a trade-off. Reasoning traces act as a "rich domain signal" that can lift general reasoning scores (MMLU-Pro), but they act as "noise" for classification, hurting the model's calibration (its ability to accurately report how certain it is).
  • Summarization as a Middle Ground: Compressing traces moves the model's performance back toward the response-only regime, suggesting that the "extra" tokens in a long CoT are indeed a source of interference for simple classification tasks.

Key Results

  • Response-only is hard to beat: It yielded the best in-domain calibration (lowest NLL and Brier scores) while keeping general capabilities very close to the frozen base model.
  • Reasoning traces are a double-edged sword: In Stage 2, only two configurations managed to outperform the frozen baseline on every task. While they provided a slight "lift" to MMLU-Pro, they lagged behind response-only models in classification accuracy.
Performance across different $\lambda$ values. Note how NLL (uncertainty) generally worsens as more trace signal is introduced.
  • Dual-adapters add complexity, not value: Splitting the rank budget between trace and label adapters yielded no significant gains over a single blended-loss adapter.
  • Summarization Trade-offs: Shorter summaries improved calibration but led to a slight drop in instruction-following (IFEval) performance.
The effect of trace summarization. Shorter traces (25%) move the model closer to the response-only baseline for NLL and Brier scores.

Limitations and Future Work

The study focused exclusively on the financial domain and the Qwen3 model family. The authors acknowledge that these dynamics might shift at different data scales or on different types of tasks (e.g., generative Q&A vs. classification).

Future research directions include:

  • Testing if attaching traces to only a subset of training examples can provide the MMLU-Pro "lift" without the cost to classification calibration.
  • Exploring different optimizers beyond AdamW.
  • Applying this staged protocol to other classification-heavy domains like legal or medical risk compliance.

Summary updated

Abstract

Fine-tuning foundation models on financial data has a wide range of applications, including identifying event types from news, monitoring risk and compliance language, and answering questions over filings and earnings calls. In this setting, the main challenge is to improve task performance without degrading the generalization capabilities of the model. In this paper, we present an extensive set of ablations for fine-tuning Qwen3-8B with low-rank adaptation (LoRA). We compare three training regimes: (1) response-only LoRA, where the supervised response is the label alone; (2) single-adapter reasoning, where reasoning traces from a teacher model are included alongside the labels in the training loss; and (3) dual-adapter composite training, where separate LoRA adapters are trained on traces and labels. Response-only LoRA improves in-domain performance and leaves general capabilities near those of the frozen model. The in-domain change is largest for calibration and argmax accuracy. Under single-adapter reasoning, the in-domain metrics remain behind those of the response-only regime, while MMLU-Pro can rise slightly and the other retention benchmarks stay within ordinary variation. Dual-adapter composite training remains on the same side of this comparison and yields no further gain. Finally, we compress the teacher traces by summarizing them before training, and show that shorter summaries move calibration toward the response-only regime without reaching it.
BibTeX
@misc{ashkboos2026reasoning,
  author = {Saleh Ashkboos and Dhruv Desai and Lavinia Ghita},
  title = {{Reasoning Supervision and Capability Retention in LoRA Domain Adaptation}},
  year = {2026},
  eprint = {2611.17072},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17072},
}
Figure