Self-Explainable Architectures for Faithful Machine Learning

This direction is merged into another direction: post-hoc explanation unfaithfulness. Its last version stays readable and citable here.

Version 1 of 189 papers94 conceptsfrozen 2026-10-04

In high-stakes domains like healthcare, finance, and autonomous driving, a model’s prediction is only as valuable as the trust we can place in it. For years, the field of Explainable AI (XAI) has relied on "post-hoc" methods—tools like LIME or SHAP that attempt to explain a black-box model after it has already been trained. However, a growing body of research suggests these approximations are often unfaithful to the model's actual logic, serving more as persuasive "marketing tools" than reliable diagnostics.

The emerging alternative is Self-Explainable AI (S-XAI): architectures that are interpretable by design. Instead of guessing why a model made a decision, these "ante-hoc" systems bake transparency into the computational path itself.

The Illusion of Post-Hoc Interpretability

The fundamental critique of post-hoc explanations, as articulated by Zachary Chase Lipton in The Mythos of Model Interpretability, is that they often prioritize human-plausibility over technical faithfulness. Because these methods are decoupled from the model’s training, they can produce rationales that are inconsistent with the actual decision-making logic.

Recent work by Zhen Tan and colleagues in Are We Merely Justifying Results ex Post Facto? identifies a phenomenon called Explanatory Inversion. They find that post-hoc explainers often work backward from the result to find a plausible justification, rather than tracing the forward path of the decision. This creates a "fidelity gap" where the explanation looks right to a human but doesn't actually reflect which features the model relied on. Maruan Al-Shedivat and others in Contextual Explanation Networks argue that this misalignment is dangerous in high-stakes scenarios, as it can hallucinate logical paths the model never took.

Prototypical Reasoning: This Looks Like That

One of the most intuitive ways to build an inherently interpretable model is through prototypical learning. Instead of mapping inputs to abstract high-dimensional vectors, these models make decisions by comparing a new input to a set of representative "prototypes" from the training data.

In ProtoAttend: Attention-Based Prototypical Learning, Sercan O Arik and Tomas Pfister demonstrate how attention mechanisms can select specific training examples to justify a prediction. This "case-based reasoning" allows a model to say, "I think this image is a bird because these specific parts look like parts of this known bird prototype."

This approach has been extended to various domains:

  • Language: Patrick Schramowski developed Proto-Trex, which allows humans to "coach" a model by refining the prototypes it uses for text classification.
  • Reinforcement Learning: Andrea Marzo introduced Prototype Soft Actor-Critic, enabling RL agents to explain their actions by referencing past states they have encountered.
  • Computer Vision: Khawla Elhadri and others survey Part-Prototype Models (PPMs), which decompose images into human-understandable visual concepts rather than opaque feature maps.

Logic and Rule-Based Regularization

Another strategy for inherent interpretability is forcing neural networks to adhere to symbolic logic or sparse rules. The goal is to maintain the predictive power of deep learning while ensuring the final decision boundary is "human-simulatable."

Michael C Hughes and his team proposed Tree Regularization in Beyond Sparsity: Tree Regularization of Deep Models for Interpretability. This technique uses a differentiable surrogate to penalize models whose decision boundaries cannot be easily approximated by a small decision tree. Similarly, Zihao Chen introduced the Rule Representation Learner (RRL), a network that can be mathematically converted into a set of "IF-THEN" rules, providing the transparency of a linear model with the performance of a deep one.

For tabular data and fact verification, models like LOREN (Jiangjie Chen) and Neural Reasoning Networks (S. Carrow) use real-valued logic to generate textual explanations that are guaranteed to be faithful because the logic is the model.

Designing for Faithfulness

The shift toward "ante-hoc" generation, as described by Akash Guna R. T. in Ante-Hoc Generation of Task-Agnostic Interpretation Maps, suggests that interpretability should not be an afterthought. By integrating specialized "weight networks" or "entropy-based layers" (Pietro Barbiero), researchers are creating systems where the explanation and the prediction are two sides of the same coin.

As Joanna Radin argues in her reflection on explainable AI competitions, we often use black-box models when we don't need to. By choosing architectures that are interpretable by design—such as the additive models proposed by Karthikeyan K or the faithful attention maps of Ananthu Aniraj—we can move toward a future where AI is not just accurate, but fundamentally accountable.

Papers in this version

Showing 60 of 89 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_self_explainable_architectures_for_faith_f9dabd17,
  title = {Self-Explainable Architectures for Faithful Machine Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-explainable-architectures-for-faithful-machine-learning/rdr_f9dabd17508d4fa097d001e2892b97bb},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_self_explainable_architectures_for_faith_f9dabd17_v1,
  title = {Self-Explainable Architectures for Faithful Machine Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-explainable-architectures-for-faithful-machine-learning/rdr_f9dabd17508d4fa097d001e2892b97bb/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}