Introduction: The Crisis of Interpretability
Mechanistic Interpretability (MI) aims to reverse-engineer neural networks into human-understandable components. However, the field currently lacks a universal "yardstick" for deciding which explanations are actually good. Researchers often encounter "Interpretability Illusions"—explanations that seem plausible to humans but fail to reflect the model's true internal logic. Without objective criteria, the community risks producing a collection of "just-so stories" rather than a rigorous science.
This paper introduces the Explanatory Virtues Framework, a rubric derived from the philosophy of science to evaluate, compare, and improve AI explanations by looking at traits like simplicity, unification, and "hard-to-varyness."
The Explanatory Virtues Framework
The authors synthesize four major philosophical perspectives into a unified framework for MI. This framework distinguishes between empirical values (how well an explanation fits the data) and theoretical values (the internal properties of the explanation itself).
The framework categorizes virtues into four pillars:
- Bayesian Virtues: Grounded in information theory, these focus on metrics like Co-explanation (explaining multiple data points simultaneously) and Unification.
- Kuhnian Virtues: These represent classic scientific values such as Simplicity, Consistency, and Fruitfulness.
- Deutschian Virtues: This introduces the concept of Hard-to-Varyness—the principle that a good explanation should be difficult to "tweak" to fit new data without breaking its core logic.
- Nomological Virtues: These evaluate whether an explanation appeals to "general laws" or universal principles of neural networks, moving beyond isolated feature catalogs.
Key Insights: From Description to Proof
A central argument of the paper is the "Mechanistic Turn": MI is evolving from a descriptive stage (cataloging neurons) to a causal stage (proving mechanisms).
- Compact Proofs as the Gold Standard: The framework identifies "Compact Proofs"—where an explanation is converted into a formal mathematical guarantee of model performance—as the most virtuous method. These proofs naturally enforce a trade-off between accuracy and simplicity.
- Simplicity vs. Parsimony: The authors clarify that "Parsimony" (fewer features) is often less valuable than "Conciseness" (shorter information-theoretic description length).
- Nomothetic Approach: Instead of "stamp collecting" (naming every feature), the authors advocate for finding universal principles that apply across different architectures.
Evaluating Current MI Methods
The paper provides a qualitative "report card" for existing techniques:
- Clustering: Scores poorly due to a lack of causal depth and failure to unify behaviors.
- Sparse Autoencoders (SAEs): Strong on simplicity and accuracy, but often produce "ad-hoc" explanations that are too easy to vary.
- Causal Circuits: Excellent at demonstrating causality but often fail at Unification; a circuit for one task rarely explains another.
- Compact Proofs: Score the highest across most virtues, providing falsifiable, hard-to-vary, and precise explanations.
Limitations and Future Directions
Despite the framework's strengths, several challenges remain:
- Scaling: Compact Proofs are difficult to scale to massive Large Language Models (LLMs) due to "superposition noise."
- The Human-AI Gap: A mathematically perfect proof might be incomprehensible to a human. The authors propose Local Decodability—the ability to understand parts of an explanation in isolation—as a future virtue.
- Defining Simplicity: A universally accepted mathematical definition for "simplicity" in neural networks remains an open research question.
Summary updated