Local Agreement Filtering for Learned Sparse Document Expansion

Learned sparse retrievers fight vocabulary mismatch, the problem that a query and a relevant document often describe the same thing in different words. They do this by "expanding" each document: a language model predicts extra vocabulary terms the document is about, and those terms go into an ordinary inverted index alongside the words actually present. Expansion is where these models win. It is also where they fail. They add spurious terms, their postings lists grow, and they become several times slower than plain keyword search. We propose a simple test for keeping an expansion term. A term earned at a word position should survive only if the model predicts it both when it reads the whole document and when it reads only a small window of neighboring words around that position. Terms that need distant context to appear are treated as likely drift and are down-weighted or dropped.

The bet rests on a linguistic finding. Speakers who use an ambiguous or vague word tend to surround it with more informative nearby context, which suggests the information needed to disambiguate a term is usually local. If that holds for retrieval text, local-window predictions should keep most useful expansions and discard much of the noise. The test costs nothing in training: it is an indexing-time intervention on an existing model, and it should shrink the index, speed up retrieval, and improve zero-shot transfer to new domains.

Core Claim

SPLADE v2 supplies the method we build on. It encodes text with BERT and uses the masked-language-model head (the layer that predicts a vocabulary distribution at each position) to score every vocabulary term at every position. It then max-pools those scores over positions into one sparse vector and distills from a cross-encoder teacher. SPRINT motivates the work. It shows that expansion outside the literal document vocabulary is the sole source of SPLADEv2's zero-shot advantage, since stripping the expanded terms drops it below uniCOIL. The same paper reports that these neural sparse models run 3.5x to 46x slower than BM25. Expansion is therefore both the asset and the cost. Nobody in this material has asked which expansion terms carry the gain.

We write $g_{i,v}$ for the weight the model assigns to vocabulary term $v$ at position $i$ under full self-attention. We write $\ell^{(k)}_{i,v}$ for the weight from the same network when attention is masked so that every layer sees only tokens within $k$ positions of $i$. The filtered document weight is

$$w_v = \max_i \; \min\left(g_{i,v},\; \ell^{(k)}_{i,v}\right),$$

so a term enters the index only with the weight that both views agree on at some single position. The claim is that this rule removes a large share of expansion mass with little loss in effectiveness on BEIR, and may even improve it, because the removed terms tend to be the ones that are not grounded locally.

The window is not arbitrary. Speakers Fill Lexical Semantic Gaps with Context measures word ambiguity from WordNet senses and from the spread of BERT embeddings across 18 languages. It finds a consistent negative correlation, from −0.31 in Portuguese to −0.55 in Yoruba, between a word's ambiguity and how uncertain its surrounding context is. That is evidence the disambiguating signal sits near the ambiguous word. The study was done on Wikipedia, so we treat its transfer to web, scientific and argument text as a hypothesis we test, not an assumption.

How This Differs From Prior Local-Context Work

Earlier work on local context used convolutions over word windows for matching or for representing word meaning, never as a check on sparse expansion. Modeling Interestingness with Deep Neural Networks builds C-DSSM, which slides convolutions over n-grams and max-pools into a dense vector, reaching NDCG@1 of 0.259 against 0.142 for BM25 on contextual entity search. Its output is dense, with no explicit terms and no inverted index. WikiQA shows that 20.3% of correct answer sentences share no content words with the question. Its sentence CNN combined with word-count features reaches 0.6520 MAP, but only as a reranker over a small candidate pool. Learning Context-Specific Word/Character Embeddings uses a convolution over context windows to create new sense vectors on the fly, reaching ρ=0.699 on the SCWS contextual word-similarity benchmark, but it targets sequence labeling.

All three use locality to build a representation. We use it to audit one: locality decides which of a strong global model's expansions to trust. CEQE contextualizes pseudo-relevance feedback terms with BERT. Pseudo-relevance feedback means running the query once, treating the top-ranked documents as relevant, and adding terms from them before searching again. CEQE finds that max-pooling over query terms beats a single averaged query vector, a hint that expansions need only align with one local component. It still contextualizes over full passages and needs several retrieval passes. Our method runs once, at indexing time.

Research Program

We start from a released DistilSPLADE-max checkpoint and implement banded attention masks for $k \in \{2, 4, 8, 16, 32\}$, plus a full-document setting. These are applied to the document encoder only, since queries are short enough that local and global views nearly coincide. We index MS MARCO and the BEIR corpora with the agreement rule.

We then analyze which terms the rule removes. We measure the fraction of expansion mass removed. We also measure an expansion-precision proxy: of the query-document score contributed by expansion terms, how much falls on relevant versus non-relevant documents according to the relevance judgments. The rule should cut the non-relevant share more than the relevant share. Next, we test the ambiguity link directly. We compute each document token's masked-prediction entropy given its window, which measures how strongly the neighbors determine the token, and ask whether expansions at well-determined positions survive more often and contribute more to relevant matches.

If the inference-only rule helps, a second stage fine-tunes the model with a training loss that adds a penalty on the gap between the global and windowed expansion weights. This makes the model produce agreeing expansions natively and removes the second encoding pass at indexing time.

Baselines

The naive reference point is BM25 over the unexpanded index. On BEIR, SPRINT reports 0.429 average nDCG@10, and BEIR itself shows BM25 beating several dense retrievers zero-shot. We run it through the same toolkit on the same corpora.

The strongest existing method is DistilSPLADE-max as published in SPLADE v2. It reports 0.500 average NDCG@10 across 13 BEIR datasets, MRR@10 of 0.368 on MS MARCO, and NDCG@10 of 0.729 on TREC DL 2019. SPRINT's re-evaluation of SPLADEv2 gives 0.470 on BEIR. We rerun the identical checkpoint ourselves, so filtered and unfiltered indexes share weights, tokenizer, query encoder and retrieval engine, and the only change is the document weights. Beating it means one of two outcomes. The first is equal or better BEIR nDCG@10, within a paired significance test per dataset, with a clearly smaller index and lower query latency. The second is a better average at equal latency.

A key ablation isolates the agreement ingredient: windowed-only weights, $\ell^{(k)}$ alone, without the minimum against the global view. A further ablation replaces the window rule with random removal of the same amount of expansion mass.

docT5query, which BEIR reports beating BM25 on 11 of 18 datasets, is included as the generative document-expansion competitor. CEQE-style contextual pseudo-relevance feedback on Robust04 and TREC DL 2019 is included as the query-side competitor. CEQE reports MAP gains of 18% and 31% over static-embedding expansion. We do not have its absolute figures from these sources, so we rerun it.

Why not just…?

Why not just split documents into short chunks and run SPLADE on each? Chunking is locality by another route, and it is a real threat. If chunked SPLADE matches our filter, the agreement rule adds nothing. We run chunked SPLADE with chunk lengths equal to $2k$, with and without overlap, at matched compute, as a direct baseline. The windowed-only ablation above is close to overlapping chunking. Our claim survives only if the minimum against the global view beats both, which would show that requiring the two views to agree, and not locality alone, is what removes noise.

Why not just raise SPLADE's sparsity regularization or keep the top-n terms per document? That shrinks postings and recovers speed without any linguistic signal. We sweep the sparsity penalty and top-n pruning to trace an effectiveness-versus-latency curve, then place the agreement rule on the same axes. If the rule lies on or below that curve, the simpler knob wins and we will report it.

Why not just use an LLM query rewriter? RaFe trains rewriters using reranker feedback, and Think Then Rewrite adds reasoning chains to bridge legal and medical jargon on EQUALS and KUAKE-IR. Both put a generative model on every query and both document drift risks. They also act on the query, whereas we act on the index at zero query-time cost, so the two approaches can be combined. We include one rewriter run on the BEIR question-answering subsets for reference and do not claim to replace it.

Why not just replace the transformer with a convolutional sparse encoder? C-DSSM shows convolutions capture phrases, but nothing in this material shows a convolutional encoder matching BERT's expansion quality. SPRINT's result that expansion quality drives performance makes a weaker encoder a poor trade. Keeping the strong model and constraining its attention keeps its knowledge while adding the locality bias.

Decisive Checks

The idea is supported if three things hold. First, at some window size $k$, the filtered index loses no more than noise-level nDCG@10 on BEIR against the rerun DistilSPLADE-max while measurably cutting postings length and latency. Second, the full agreement rule beats both chunked SPLADE and the windowed-only ablation. Third, the removed expansion mass falls disproportionately on non-relevant documents. A supporting check is that on the zero-overlap WikiQA questions, filtered expansion recovers answers that BM25 misses about as well as unfiltered expansion does.

The idea is refuted if effectiveness falls steadily as $k$ shrinks while chunking or plain pruning matches every gain. In that case expansion noise is not local, and the Pimentel et al. trade-off does not carry over to retrieval text.