TL;DR
Dense retrieval models suffer from a pronounced "positional bias," frequently ignoring relevant information if it appears later in a document. This paper introduces a zero-shot, inference-time attention calibration technique that redistributes attention mass more evenly across text segments. By introducing a tunable strength coefficient ($\lambda$), the authors mitigate position bias across multiple architectures and languages without requiring any retraining or sacrificing overall retrieval effectiveness.
What Problem Does This Paper Address?
Modern dense retrieval models map queries and documents into a shared embedding space to perform semantic matching. However, these models exhibit a frustrating flaw: they are heavily front-loaded. If the answer to a query lies near the beginning of a passage, the model finds it easily; if it lies in the middle or at the end, performance plummets.
This positional bias happens because the "pooling token" (like <s> or <CLS>), which aggregates the final document representation, disproportionately focuses its attention weights on early tokens during the forward pass. While previous work attempted to fix this by completely flattening representation profiles, doing so often degrades downstream retrieval performance. The challenge lies in reducing positional bias without destroying the model's overall retrieval capabilities and without the massive computational expense of retraining the model from scratch.
Methods
The authors propose an inference-time intervention called Attention Calibration with a Strength Coefficient.
Instead of letting the pooling token focus heavily on the first few words, the method chunks the document's token sequence into contiguous "baskets" of a fixed size ($\mathfrak{B}$). It then enforces equal total attention mass across all baskets while preserving the relative attention weights within each basket.
The core novelty is the introduction of a strength coefficient, $\lambda \in [0, 1]$, which allows for partial calibration. It linearly interpolates between the model's original attention distribution ($\alpha$) and the fully equalized distribution ($\hat{\alpha}_{cal}$):
$$\hat{\alpha} = \lambda \cdot \hat{\alpha}_{cal} + (1 - \lambda) \cdot \alpha$$
Setting $\lambda = 0$ leaves the model untouched, while $\lambda = 1.0$ applies full calibration. This intervention is applied entirely at inference time to the query row of the pooling token in selected transformer layers (e.g., the last 50% of layers), requiring no gradient computation or retraining.
Datasets
The approach was rigorously evaluated across three position-aware benchmarks:
- SQuAD-PosQ: A modified version of SQuAD v2 containing short passages (averaging ~150 subtokens), where queries are grouped by the character position of the answer.
- FineWeb-PosQ: A longer-form dataset (~768 subtokens) derived from FineWeb-edu, where passages are split into beginning, middle, and end segments with section-specific questions.
- Position Aware IR Benchmark (PosIR): A massive multilingual benchmark spanning 10 languages, 31 domains, and 4 document-length quartiles (from 512 to 2048 tokens), used to test large-scale generalization.
Key Insights and Ideas
- Partial Calibration is the Sweet Spot: Completely flattening the attention distribution ($\lambda = 1.0$) actually hurts retrieval performance. It turns out that some amount of the original, front-loaded attention is necessary to preserve semantic meaning. The authors found that a moderate setting ($\lambda = 0.5$) consistently yields the best trade-off.
- One Configuration to Rule Them All: A single default configuration—basket size $\mathfrak{B} = 128$, strength $\lambda = 0.5$, applied to the last 50% of layers—improved performance across three entirely different embedding models without any per-model tuning.
- Architecture Agnostic: The method works equally well regardless of whether the model uses prefix pooling (
<s>-pooling) or postfix pooling (last-token pooling). You simply calibrate the query row corresponding to whichever token acts as the aggregator.
Key Results
The authors used the Position Sensitivity Index (PSI)—which measures the performance gap between the best and worst positional groups—and the harmonic mean of nDCG@10 to evaluate their method.
- FineWeb-PosQ: The default configuration ($\lambda = 0.5$) improved the harmonic mean of nDCG@10 for all three evaluated models (gte-multilingual-base, bge-m3-dense, and Qwen3-Embedding-0.6B). For Qwen3, performance on the hardest "end" segment jumped from 65.61 to 75.63 nDCG@10.
- PosIR Generalization: When transferred blindly to the massive PosIR benchmark, the default configuration reduced positional sensitivity (PSI) in all 16 combinations of document lengths, models, and retrieval settings (multilingual and cross-lingual), while simultaneously maintaining or boosting aggregate nDCG@10.
- Beating the Baseline: It significantly outperformed a standard chunk-and-average baseline (Segment Embed Average). While chunking also reduced position bias, it tanked overall retrieval performance (dropping bge-m3-dense by over 10 nDCG points), whereas attention calibration kept overall effectiveness flat or positive.
Limitations and Future Work
- Dense Retrieval Only: This work is strictly limited to bi-encoder dense retrieval models. How attention calibration interacts with cross-encoders, re-rankers, or hybrid lexical-dense systems remains unexplored.
- Coarse Parameter Grid: The "default" configuration was found using a relatively small grid search. A joint global optimization over basket size, layer depth, and lambda might yield even better results but is currently prohibitively expensive to compute.
- Lack of Mechanistic Explanation: While the empirical results are strong, the paper does not provide a deep mechanistic explanation of why partial calibration strikes the best balance, or how it fundamentally interacts with complex learned attention patterns and positional encodings.
Summary updated