Towards Analogy-Based Explanations in Machine Learning

Eyke Hüllermeier

Abstract

Current explainable AI (XAI) methods predominantly rely on similarity-based justifications, which often fail when local data is sparse or when explaining relational preferences. This paper proposes a novel framework for analogy-based explanations, shifting the paradigm from "this is like that" to "A is to B as C is to D." By formalizing analogical proportions in both Boolean and continuous spaces, the author demonstrates how analogies can provide more robust and intuitive justifications for classification and ranking tasks, particularly by leveraging a larger combinatorial pool of potential explanations across the feature space.

Introduction

As machine learning models—especially deep neural networks—become more autonomous and "black-box" in nature, the demand for Explainable AI (XAI) has skyrocketed. Most current "example-based" explanations rely on similarity: if a model predicts a journal belongs in "Category A," it might justify this by showing the most similar journal already in that category.

However, similarity-based explanations have significant flaws. First, "similarity" is difficult to quantify—determining how close is "close enough" is often arbitrary. Second, these explanations are strictly local; if there are no similar examples in the training data near the query point, the explanation fails. Finally, similarity is often a poor fit for ranking or preference learning, where the relationship between items is more critical than the items' individual attributes.

Methods: The Mechanics of Analogy

The paper introduces analogy-based explanations as a complement to similarity. The core mechanism is the analogical proportion, denoted as $a : b :: c : d$ (read as "$a$ is to $b$ as $c$ is to $d$").

The author formalizes this using two primary approaches:

  1. Boolean Logic: For binary features, an analogy holds if the pattern of changes between $(a, b)$ matches $(c, d)$.
  2. Numerical Arithmetic: In a continuous space, an analogy is defined by the "relatedness" of feature vectors. For example, using the arithmetic difference: $v(a, b, c, d) = 1 - |(a - b) - (c - d)|$. If the difference between $a$ and $b$ is the same as the difference between $c$ and $d$, the analogy is strong.
Analogical proportion and transfer: if $a$ relates to $b$ as $c$ relates to $d$, the label for $d$ can be inferred.

A critical component is analogical transfer. If we know $a$ is preferred over $b$, and we find that $a : b :: c : d$, we can hypothesize that $c$ is preferred over $d$. For explanations, instead of just showing a neighbor, the system presents a triplet $(a, b, c)$ that forms an analogy with the query $d$. This explains the prediction by showing a consistent "transformation" or "relationship" that exists elsewhere in the data.

Experimental Setup

The paper illustrates its concepts using a real-world dataset of 172 scientific journals in the field of pure mathematics. Journals are scored on five criteria: total citations, Impact Factor (IF), Immediacy Index (II), number of articles, and cited half-life. These journals are classified into quality categories ($A^*, A, B, C$). The task involves using these features to predict a journal's category or to rank journals against one another.

Key Insights

  • Analogy as a Relaxation of Similarity: Similarity requires two points to be close in space. Analogy only requires the difference between two pairs of points to be similar. This allows the explainer to pull "evidence" from parts of the feature space that are far away from the query point, overcoming the limitations of local density.
Analogy-based explanations can leverage triplets $(a, b, c)$ from across the feature space to justify a prediction for query $d$.
  • The Combinatorial Advantage: For a query point $d$, there is only a linear number of potential "nearest neighbors" to use for a similarity explanation. However, there is a cubic number of potential triplets $(a, b, c)$ that could form an analogy. This provides a much larger pool of potential explanations.
  • Natural Fit for Preferences: Explaining why "Journal C is better than Journal D" is easier if you can say, "It's the same reason Journal A is better than Journal B." This captures the logic of the ranking rather than just the attributes of the items.

Key Results

The paper provides a compelling qualitative result using the journal dataset. In a "step-down" explanation, a journal $d$ classified as Category C might have a confusing similarity explanation if it looks somewhat like a Category B journal. However, an analogy-based explanation can find three other journals where an A-journal relates to a B-journal exactly as a B-journal relates to the query $d$. This clarifies that the "gap" in quality is consistent with a known drop from B to C.

Quantitative analysis further demonstrates that the number of valid analogies remains high even when the degree of similarity to the query point drops. This proves that analogy-based systems have more "explanatory material" to work with than similarity-based ones, especially in sparse regions of the feature space.

Limitations and Future Work

Despite the theoretical promise, several challenges remain. The paper does not yet define a formal "best" way to select which analogy to present to a user among the thousands of possibilities. Furthermore, user studies are required to confirm if humans find these four-part analogies more intuitive than simple similarities. Finally, searching through triplets ($O(n^3)$) is significantly more computationally expensive than searching for neighbors ($O(n)$), necessitating the development of more efficient search algorithms for large-scale datasets.

Summary updated

Abstract

Principles of analogical reasoning have recently been applied in the context of machine learning, for example to develop new methods for classification and preference learning. In this paper, we argue that, while analogical reasoning is certainly useful for constructing new learning algorithms with high predictive accuracy, is is arguably not less interesting from an interpretability and explainability point of view. More specifically, we take the view that an analogy-based approach is a viable alternative to existing approaches in the realm of explainable AI and interpretable machine learning, and that analogy-based explanations of the predictions produced by a machine learning algorithm can complement similarity-based explanations in a meaningful way. To corroborate these claims, we outline the basic idea of an analogy-based explanation and illustrate its potential usefulness by means of some examples.
BibTeX
@inproceedings{hullermeier2020towards,
  author = {Eyke Hüllermeier},
  title = {{Towards Analogy-Based Explanations in Machine Learning}},
  booktitle = {MDAI},
  year = {2020},
  doi = {10.1007/978-3-030-57524-3\_17},
}
Figure