The Faithfulness Gap and the Shift to Intrinsic Explanation
Traditional Explainable AI (XAI) often relies on post-hoc methods—such as feature attribution maps or separate explanation models—to rationalize a decision after it has been made. However, as Ras et al. note in their taxonomy of explainable deep learning (Explainable Deep Learning: A Field Guide for the Uninitiated), post-hoc rationalizations can create a "faithfulness gap" where the generated explanation sounds plausible but fails to reflect the model's true internal logic. Indeed, Atanasova et al. demonstrated in Faithfulness Tests for Natural Language Explanations that decoupled, post-hoc models can generate unfaithful explanations up to 60% of the time.
To bridge this gap, researchers have turned to joint task and explanation training. Instead of treating prediction and justification as a two-step pipeline, models are optimized to perform both tasks simultaneously. By forcing a single architecture to output both a categorical label and a natural language rationale, the explanation becomes computationally and intrinsically linked to the decision-making process.
Architectural Paradigms: Multi-Head vs. Unified Sequences
How do models learn to predict and explain at the same time? Two primary architectural strategies dominate this space:
- Multi-Task Shared Encoders: In this setup, a single backbone encoder processes the input and passes its latent representations to multiple specialized output heads. For instance, in Self-training with Few-shot Rationalization, Bhat et al. utilize a shared encoder where one head extracts token-level rationales and another classifies the label. Similarly, in the robotics domain, Sachdeva et al. design a framework (Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning) that shares a common feature representation to concurrently rank traffic agent importance and generate natural language reasoning. This multi-task coupling ensures that the generated explanations are logically consistent with the predicted priority levels.
- Unified Sequence Generation: With the rise of text-to-text transformers, a simpler yet highly effective approach is to frame both prediction and explanation as a single contiguous text sequence. Narang et al. demonstrated this paradigm with WT5 (WT5?! Training Text-to-Text Models to Explain their Predictions), showing that models can be trained to output a label and its supporting explanation sequentially within a single string. As explored by Marasovic in Few-Shot Self-Rationalization with Natural Language Prompts, this mechanism allows the model to justify its classification decisions using the exact same representational space used for the prediction itself. In multimodal settings, models like NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks format answer prediction and natural language explanation as a single generative sequence, eliminating the need for separate task models and reducing resource overhead.
Why It Works: Explanation as a Regularizer
A common assumption in machine learning is that adding interpretability constraints degrades a model's primary task accuracy. Joint rationale generation flips this assumption on its head: learning to explain frequently improves predictive performance. This synergy occurs due to two main mechanisms:
- Causal Supervision and Feature Grounding: Training a model to articulate why it made a choice acts as a form of causal supervision. For example, Xu et al. found in Explainable Object-Induced Action Decision for Autonomous Vehicles that forcing an autonomous vehicle model to explain its driving commands encourages the network to learn the underlying logic connecting environmental states to specific behavioral outputs. This prevents the model from latching onto spurious correlations. Similarly, Hagos et al. leverage this in Unlearning Spurious Correlations in Chest X-ray Classification by combining standard predictive loss with an auxiliary feature attribution loss, forcing medical imaging models to ignore misleading shortcuts (like shoulder bones) and focus on clinically relevant regions.
- Representation Regularization: The secondary task of generating human-readable text provides a structured internal representation that regularizes the primary task. Li et al. showed in A Joint Learning Framework for Restaurant Survival Prediction and Explanation that jointly optimizing for binary outcome prediction and natural language explanation generation yields mutual performance gains, as the explanatory task aligns the model's internal logic with human-understandable evidence. Sharma et al. observed a similar effect in What Do You MEME? Generating Explanations for Visual Semantic Role Labelling in Memes, noting that a joint-loss objective enhances classification stability and accuracy by leveraging the inherent complementarity between a categorical label and its justification.
Cross-Domain Generalization and Robustness
The benefits of joint prediction and explanation extend across diverse domains and learning settings:
- Mitigating Catastrophic Forgetting: In continual learning scenarios, models often struggle to distinguish between highly similar tasks over time. Xiong et al. demonstrated in Rationale-Enhanced Language Models are Better Continual Relation Learners that forcing a model to generate and utilize rationales helps it internalize the underlying logic of a task, significantly reducing forgetting when learning new relations.
- Complex Domain Reasoning: From legal charge prediction (Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions by Ye et al.) to pharmacology (ExDDI: Explaining Drug-Drug Interaction Predictions with Natural Language by Sun et al.), requiring a model to output intermediate clinical descriptors or step-by-step reasoning chains (HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning by Yang et al.) ensures that the final output is grounded in a verifiable, human-interpretable path.
Key Foundational Work
e-QRAQ: A Multi-turn Reasoning Dataset and Simulator with Explanations (2017) — contributed 1 concept elements to this direction.
e-SNLI: Natural Language Inference with Natural Language Explanations (2018) — contributed 1 concept elements to this direction.
Interpretable Rationale Augmented Charge Prediction System (2018) — contributed 1 concept elements to this direction.