NormXLogit: The Head-on-Top Never Lies

Sina Abbasi, Mohammad Reza Modarres, Mohammad Taher PilehvarEMNLP 2025

TL;DR

This paper introduces NormXLogit, a novel, architecture-agnostic method for interpreting large language models (LLMs) by analyzing the "head-on-top" output layer. NormXLogit efficiently identifies important input tokens by combining the norm of word embeddings with logit attributions, demonstrating superior faithfulness and computational efficiency compared to existing methods.

What Problem Does This Paper Address?

Interpreting the decision-making process of complex Transformer-based LLMs is crucial for understanding their behavior, debugging errors, and ensuring responsible deployment. While many interpretability methods exist, they often suffer from high computational costs, model-specific designs, or vulnerabilities to adversarial attacks. Existing approaches like gradient-based methods can be computationally intensive, and vector-based methods are often tied to specific architectures. There's a need for a general, efficient, and faithful method to pinpoint which input tokens are most influential for a model's predictions.

Methods

NormXLogit is an architecture-agnostic interpretation technique that leverages two key ideas:

  • Norm of Word Embeddings: The authors build on prior work showing that the L2 norm of input embeddings encodes information gain, suggesting that tokens with higher norms carry more information.
  • Head-on-Top (HoT) Logit Attribution (LogAt): This method uses the final layer's output (the "head-on-top") to attribute the model's prediction to individual input tokens. For classification tasks, it analyzes the logits corresponding to the predicted class. For regression, it uses the absolute difference from the predicted value.

NormXLogit combines these by multiplying the LogAt attribution score for a token by the L2 norm of its input word embedding: AttNormXLogit(Xi) = ||xi||2 × AttLogAt(xi). This approach avoids backward passes and complex aggregation methods like "rollout," making it computationally efficient.

Datasets

The paper evaluates NormXLogit on several standard NLP datasets:

  • Classification Tasks:
    • SST-2 (Stanford Sentiment Treebank v2) for sentiment analysis.
  • MultiNLI (Multi-Genre Natural Language Inference) for textual entailment.
  • QNLI (Question Natural Language Inference) for question answering.
  • Regression Task:
    • STS-B (Semantic Textual Similarity Benchmark) for semantic similarity.
  • Language Modeling Task:
    • BLiMP (Linguistic Minimal Pairs) for evaluating evidence alignment in masked language modeling.

Key Insights and Ideas

  • The "Head-on-Top" is Key: The authors argue that the final output layer (head-on-top) is crucial for generating task-specific explanations and that its output directly reflects token contributions.
  • Norm of Embeddings as a Signal: The L2 norm of word embeddings is a surprisingly effective proxy for token importance, especially when combined with logit attributions.
  • Architecture Agnosticism is Achievable: NormXLogit demonstrates that a powerful and faithful interpretation method can be developed without being tied to specific model architectures like BERT.
  • Efficiency Matters: The proposed method is significantly faster and more memory-efficient than many existing techniques, making it scalable for large models and long sequences.
  • Comprehensiveness vs. Sufficiency Trade-off: NormXLogit excels at comprehensiveness (identifying tokens necessary for a prediction) but is less effective at sufficiency (tokens that can preserve a prediction in isolation), which the authors attribute to the contextual nature of LLM predictions.

Key Results

  • Faithfulness: NormXLogit consistently outperforms gradient-based methods in faithfulness metrics (Confidence Drop and Accuracy) across various models and datasets. It also shows competitive performance against architecture-specific methods like DecompX.
  • Computational Efficiency: NormXLogit is significantly faster (up to 6x) and more memory-efficient (up to 750x) than other methods, especially for longer input sequences. It can handle much larger batch sizes.
  • Evidence Alignment: In the BLiMP dataset experiment, NormXLogit showed strong alignment with human-understood linguistic rationales, particularly in higher layers, indicating its ability to capture contextualized meaning.
  • Per-Label Attributions: The LogAt component allows NormXLogit to generate explanations for each specific label, providing richer insights than global attribution methods.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that, like all interpretability methods, NormXLogit lacks a definitive "gold standard" for evaluation.
  • NormXLogit is less effective at sufficiency compared to comprehensiveness, as LLM predictions often rely on contextual interactions rather than isolated tokens.
  • Future Work:
    • Explore the applicability of NormXLogit to other domains (e.g., vision) and non-Transformer architectures.
  • Investigate methods for aggregating per-label attributions to create more comprehensive explanations for classification tasks.

Summary updated

BibTeX
@inproceedings{abbasi2025normxlogit,
  author = {Sina Abbasi and Mohammad Reza Modarres and Mohammad Taher Pilehvar},
  title = {{NormXLogit: The Head-on-Top Never Lies}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure