In complex reasoning tasks—like multi-hop question answering or legal document analysis—large language models often suffer from a "shortcut" problem. Instead of actually tracing the logic required to reach a conclusion, they find spurious correlations in the training data that allow them to guess the correct answer for the wrong reasons.
To combat this, a growing body of research suggests that we should not treat the final answer as the only objective. Instead, by using multi-task evidence scaffolding, researchers are training models to simultaneously predict the answer and identify the specific evidence or reasoning steps required to get there. This joint optimization doesn't just make models more interpretable; it fundamentally changes how they learn to reason.
The Shortcut Problem and the Search for Latent Paths
When a model is trained only on (Question, Answer) pairs, the "reasoning path" between the two is a latent variable. In a massive dataset, the model might find that certain keywords in a question often appear near certain types of answers, allowing it to bypass the actual logic of the task.
Dheeru Dua and Matt Gardner demonstrated in Benefits of Intermediate Annotations in Reading Comprehension that providing even a small amount of "intermediate annotations"—labels for the specific text spans needed for reasoning—can significantly boost performance. By supervising these intermediate steps, we force the model to ground its reasoning in the source text, preventing it from adopting incorrect latent paths that might accidentally yield the correct answer during training.
Joint Optimization as a Regularizer
The core mechanism of this research direction is the multi-task loss function. Rather than a pipeline where one model finds evidence and another answers, these systems use shared representations to perform both tasks at once.
In the Select, Answer and Explain (SAE) system, Ming Tu and Jing Huang show that integrating answer prediction and evidence identification creates a feedback loop. The task of identifying evidence acts as a regularizer for the answering task, ensuring the model's internal features are focused on functionally relevant information. Similarly, the Query Focused Extractor (QFE) developed by Kosuke Nishida and colleagues uses a sequential summarization approach to ensure that extracted evidence is not just relevant to the query, but sufficient to answer it.
Decomposing the Reasoning Chain
Scaffolding can be applied to more than just text spans; it can be used to supervise the entire logical structure of a task. This is seen across several domains:
- Document-Level Relation Extraction: The SAIS framework by Yuxin Xiao and Jiawei Han supervises four distinct sub-tasks: coreference resolution, entity typing, and two levels of evidence retrieval. This ensures the model learns the underlying logic of the domain rather than just pattern matching.
- Personality Detection: In PQ-Net, Feifan Yang and Xiaojun Quan improve personality detection by forcing the model to "answer" psychological questionnaires as an intermediate step, aligning the model's reasoning with established theoretical frameworks.
- Legal Reasoning: Christopher D Manning and Yuta Koreeda applied this to ContractNLI, where the model must simultaneously determine a legal conclusion and pinpoint the exact "evidence spans" in a contract that support it.
Breaking the Performance-Explainability Trade-off
Traditionally, researchers assumed that making a model explainable would come at the cost of accuracy. Multi-task scaffolding suggests the opposite.
In the RExC framework, Bodhisattwa Prasad Majumder and Oana-Maria Camburu demonstrate that grounding explanations in external knowledge and optimizing them alongside the primary task actually improves accuracy. By requiring the model to generate both extractive rationales (highlights) and natural language justifications, the framework ensures the model's "self-rationalization" is faithful to its decision-making process.
This direction is perhaps best summarized by the QASPER dataset work by Pradeep Dasigi and Noah A. Smith. By treating evidence as a "scaffold" for the answer, we move away from black-box predictions toward systems that are verifiable by design. When a model is trained to show its work, the "work" becomes the foundation upon which the correct answer is built.