Listwise Feature Attribution for Learning to Rank

This direction is merged into another direction: feature independence assumption (feature correlation bias). Its last version stays readable and citable here.

Version 1 of 114 papers18 conceptsfrozen 2026-10-04

In traditional machine learning, interpretability is often a "pointwise" problem. If a model classifies an image as a cat, we ask which pixels contributed to that specific label. However, in Information Retrieval (IR) and recommendation systems, the output is not a single label but an ordered list. Explaining why a document is ranked at position #1 instead of #10 is fundamentally different from explaining why it was retrieved at all.

Recent research has shifted from explaining individual scores to explaining the competitive dynamics of a ranked list. This direction, often termed listwise feature attribution, seeks to uncover the features that govern the relative preferences between items.

The Failure of Pointwise Interpretability

Standard interpretability frameworks like LIME or SHAP were designed for classification and regression. As noted by Avishek Anand and Jaspreet Singh in their work on EXS, these "pointwise" explainers evaluate documents in isolation.

This creates a fidelity gap. In a ranking model, a document’s score only matters in relation to the scores of its neighbors. If a feature change increases the scores of all documents equally, the ranking remains unchanged. A pointwise explainer might incorrectly credit that feature for the document's position, whereas a listwise explainer would recognize that the feature did not drive the relative ordering. Maria Heuss and Avishek Anand argue in RankingSHAP that features common to all items in a set should not be credited with explaining the specific order of results.

Capturing Competition through Listwise Dynamics

To solve this, researchers have developed frameworks that treat ranking as a zero-sum competition. Instead of asking "What makes this document relevant?", they ask "What makes this document more relevant than the others?"

The LISTEN framework, introduced by Maartje ter Hoeve and Maarten de Rijke in Faithfully Explaining Rankings in a News Recommender System, exemplifies this shift. LISTEN identifies feature importance by analyzing how perturbations to an item's attributes affect its relative rank across the entire candidate set. Because calculating these permutations for every item in a real-time system (like news) is computationally expensive, they developed Q-LISTEN, a neural proxy that learns to predict these explanations instantly.

Axiomatic Foundations for Ranking

As the field matured, the focus moved toward mathematical rigor. If we are to trust an explanation of a search result, that explanation must follow certain logical rules (axioms).

Tanya Chowdhury and Yair Zick addressed this in RankSHAP. They adapted the game-theoretic Shapley value—a method for fairly distributing "payouts" (importance) among "players" (features)—specifically for the ranking context. By establishing ranking-specific axioms, RankSHAP ensures that feature attributions are consistent: if a feature contributes more to the relative movement of a document than another feature, its attribution score must reflect that.

This axiomatic approach solves a key problem in Learning-to-Rank (LTR): the interdependence of items. In LTR, the model's "decision" is the permutation of the list. RankSHAP quantifies how much each feature contributed to that specific permutation, rather than just the underlying scalar scores.

From Theory to Tooling

The transition from "black box" ranking to transparent IR is being accelerated by standardized libraries. Sourav Saha and colleagues introduced ir_explain, a Python library that consolidates these listwise methods. This allows researchers to move beyond simple keyword matching and interrogate complex Neural Ranking Models (NRMs) and Large Language Models (LLMs).

Similarly, work by Maria Lymperaiou and Giorgos Stamou on Conceptual Contrastive Edits shows how we can "stress test" retrieval models like CLIP or SBERT. By making mathematically optimal edits to inputs, they reveal what specific concepts cause a model to swap the positions of two items, providing a "contrastive" explanation that aligns with how humans naturally reason: "Why X instead of Y?"

Why It Works: The Intuition of Relativity

The core intuition behind this entire research direction is that context is everything.

  1. Pointwise methods see a document in a vacuum.
  2. Listwise methods see a document as a competitor in a race.

By focusing on the features that cause documents to overtake one another, these methods provide explanations that are more "faithful" to the actual logic of the ranking model. Whether through the lens of game theory (RankSHAP), local perturbations (EXS), or sentiment alignment (SAER), the goal is the same: to turn the opaque "sorted list" into a transparent set of trade-offs.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_listwise_feature_attribution_for_learnin_4a55d13b,
  title = {Listwise Feature Attribution for Learning to Rank},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/listwise-feature-attribution-for-learning-to-rank/rdr_4a55d13b21e444c8bcb4e657674797a7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_listwise_feature_attribution_for_learnin_4a55d13b_v1,
  title = {Listwise Feature Attribution for Learning to Rank},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/listwise-feature-attribution-for-learning-to-rank/rdr_4a55d13b21e444c8bcb4e657674797a7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}