Attention Calibration for Position-Fair Dense Information Retrieval

Andrianos Michail, Elias Schuhmacher, Juri Opitz, Simon Clematide, Rico SennricharXiv 2026 · arXiv

TL;DR

Dense retrieval models suffer from a pronounced "positional bias," frequently ignoring relevant information if it appears later in a document. This paper introduces a zero-shot, inference-time attention calibration technique that redistributes attention mass more evenly across text segments. By introducing a tunable strength coefficient ($\lambda$), the authors mitigate position bias across multiple architectures and languages without requiring any retraining or sacrificing overall retrieval effectiveness.

What Problem Does This Paper Address?

Modern dense retrieval models map queries and documents into a shared embedding space to perform semantic matching. However, these models exhibit a frustrating flaw: they are heavily front-loaded. If the answer to a query lies near the beginning of a passage, the model finds it easily; if it lies in the middle or at the end, performance plummets.

This positional bias happens because the "pooling token" (like <s> or <CLS>), which aggregates the final document representation, disproportionately focuses its attention weights on early tokens during the forward pass. While previous work attempted to fix this by completely flattening representation profiles, doing so often degrades downstream retrieval performance. The challenge lies in reducing positional bias without destroying the model's overall retrieval capabilities and without the massive computational expense of retraining the model from scratch.

Methods

The authors propose an inference-time intervention called Attention Calibration with a Strength Coefficient.

Instead of letting the pooling token focus heavily on the first few words, the method chunks the document's token sequence into contiguous "baskets" of a fixed size ($\mathfrak{B}$). It then enforces equal total attention mass across all baskets while preserving the relative attention weights within each basket.

The core novelty is the introduction of a strength coefficient, $\lambda \in [0, 1]$, which allows for partial calibration. It linearly interpolates between the model's original attention distribution ($\alpha$) and the fully equalized distribution ($\hat{\alpha}_{cal}$):

$$\hat{\alpha} = \lambda \cdot \hat{\alpha}_{cal} + (1 - \lambda) \cdot \alpha$$

Setting $\lambda = 0$ leaves the model untouched, while $\lambda = 1.0$ applies full calibration. This intervention is applied entirely at inference time to the query row of the pooling token in selected transformer layers (e.g., the last 50% of layers), requiring no gradient computation or retraining.

Figure 1: Attention calibration with strength coefficient λ. Top: The pooling token’s original, front-loaded atten- tion. Bottom: The calibrated distribution after basket- level equalization. The coefficient λ ∈[0, 1] linearly interpolates between them.

Datasets

The approach was rigorously evaluated across three position-aware benchmarks:

  1. SQuAD-PosQ: A modified version of SQuAD v2 containing short passages (averaging ~150 subtokens), where queries are grouped by the character position of the answer.
  2. FineWeb-PosQ: A longer-form dataset (~768 subtokens) derived from FineWeb-edu, where passages are split into beginning, middle, and end segments with section-specific questions.
  3. Position Aware IR Benchmark (PosIR): A massive multilingual benchmark spanning 10 languages, 31 domains, and 4 document-length quartiles (from 512 to 2048 tokens), used to test large-scale generalization.
Figure 2: Evaluation segments and attention calibration baskets for SQuAD-PosQ (a) and FineWeb-PosQ (b). Baskets partition the token sequence independently of evaluation groupings; B determines the size of baskets and the granularity of redistribution.

Key Insights and Ideas

  • Partial Calibration is the Sweet Spot: Completely flattening the attention distribution ($\lambda = 1.0$) actually hurts retrieval performance. It turns out that some amount of the original, front-loaded attention is necessary to preserve semantic meaning. The authors found that a moderate setting ($\lambda = 0.5$) consistently yields the best trade-off.
  • One Configuration to Rule Them All: A single default configuration—basket size $\mathfrak{B} = 128$, strength $\lambda = 0.5$, applied to the last 50% of layers—improved performance across three entirely different embedding models without any per-model tuning.
  • Architecture Agnostic: The method works equally well regardless of whether the model uses prefix pooling (<s>-pooling) or postfix pooling (last-token pooling). You simply calibrate the query row corresponding to whichever token acts as the aggregator.
Figure 3: Absolute change in nDCG@10 on FineWeb-PosQ relative to the uncalibrated baseline as a function of attention calibration strength (λ).

Key Results

The authors used the Position Sensitivity Index (PSI)—which measures the performance gap between the best and worst positional groups—and the harmonic mean of nDCG@10 to evaluate their method.

  • FineWeb-PosQ: The default configuration ($\lambda = 0.5$) improved the harmonic mean of nDCG@10 for all three evaluated models (gte-multilingual-base, bge-m3-dense, and Qwen3-Embedding-0.6B). For Qwen3, performance on the hardest "end" segment jumped from 65.61 to 75.63 nDCG@10.
  • PosIR Generalization: When transferred blindly to the massive PosIR benchmark, the default configuration reduced positional sensitivity (PSI) in all 16 combinations of document lengths, models, and retrieval settings (multilingual and cross-lingual), while simultaneously maintaining or boosting aggregate nDCG@10.
  • Beating the Baseline: It significantly outperformed a standard chunk-and-average baseline (Segment Embed Average). While chunking also reduced position bias, it tanked overall retrieval performance (dropping bge-m3-dense by over 10 nDCG points), whereas attention calibration kept overall effectiveness flat or positive.

Limitations and Future Work

  • Dense Retrieval Only: This work is strictly limited to bi-encoder dense retrieval models. How attention calibration interacts with cross-encoders, re-rankers, or hybrid lexical-dense systems remains unexplored.
  • Coarse Parameter Grid: The "default" configuration was found using a relatively small grid search. A joint global optimization over basket size, layer depth, and lambda might yield even better results but is currently prohibitively expensive to compute.
  • Lack of Mechanistic Explanation: While the empirical results are strong, the paper does not provide a deep mechanistic explanation of why partial calibration strikes the best balance, or how it fundamentally interacts with complex learned attention patterns and positional encodings.

Summary updated

Abstract

Dense retrieval models exhibit positional bias: retrieval effectiveness degrades when relevant information appears later in a passage (Zeng et al., 2025). We ask whether this bias can be reduced at inference time, without retraining and without sacrificing overall retrieval effectiveness. To this end, we adapt inference-time attention calibration (Schuhmacher et al., 2026) to downstream retrieval and extend it with a strength coefficient lambda that interpolates between the original and fully calibrated attention distributions. Across three embedding models on SQuAD-PosQ and FineWeb-PosQ, we examine how basket size, calibrated layer set, and strength affect the trade-off between positional fairness and retrieval effectiveness, finding that partial calibration frequently outperforms full calibration. A single configuration (B=128, lambda=0.5, 50% layer depth) improves the harmonic mean of nDCG@10 across positional groups on FineWeb-PosQ for all three models without per-model tuning, and applies to both <s>-pooled and last-token-pooled architectures. This default configuration transfers without modification to PosIR, which spans 10 languages and 31 domains, reducing the Position Sensitivity Index in all 16 length-quartile x model x retrieval-setting combinations, while preserving or improving aggregate nDCG@10. We release our extended codebase at https://github.com/impresso/fair-sentence-transformers
BibTeX
@misc{michail2026attention,
  author = {Andrianos Michail and Elias Schuhmacher and Juri Opitz and Simon Clematide and Rico Sennrich},
  title = {{Attention Calibration for Position-Fair Dense Information Retrieval}},
  year = {2026},
  eprint = {2606.02737},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.02737},
}

Loading related content...

Figure