Pooling Sparse Click Feedback Across Devices to Close the Vocabulary Gap

Search fails most often when the person asking and the person who wrote the document use different words for the same thing. A trainee types "the green plug for DIN rails" and the catalog says "terminal block, push-in connection." This is the vocabulary mismatch problem, also called the lexical gap. The evidence that closes such gaps is sitting in logs: which document a user actually opened after typing which words. In private or on-device settings, that evidence is scattered. Each phone, clinic, or small shop holds a handful of query–click pairs, far too few to learn any word mapping locally, and the raw logs cannot be sent to a server. We propose that devices share only clipped, noised counts of which query words led to which document words. A server sums these counts into one translation table, uses it to add user vocabulary to documents at indexing time, and each device blends the shared table with its own few observations.

The payoff is a retrieval system that learns the vocabulary of its actual user community without collecting a single raw query and without running a neural model on the device. At query time it remains a keyword index, as cheap as BM25, the standard term-matching ranker. This matters because the tools that currently close the lexical gap make things worse in exactly this setting. Learned expansion models are trained on public web data and do not know community jargon. Generative query rewriters are slow, need cloud calls, and can erase technical terms. Federated recommenders that handle sparse clients work on item IDs, not words.

Core Claim

When the vocabulary a community uses is specific to that community, the signal needed to bridge it is spread thinly across many sparse clients. Summing lexical feedback across clients with a privacy guarantee recovers most of the benefit of a centralized log, and this benefit is out of reach for any single client and for public expansion models. Put formally, let client $c$ hold $n_c$ accepted (query, document) pairs. Each pair contributes a normalized co-occurrence vector over (query term $a$, document term $b$). The client sums these, clips the total to norm $C$, and adds Gaussian noise. The server receives only the noised sum $\tilde w_c(a,b)$ through secure aggregation and forms

$$T(a \mid b) = \frac{\sum_c \tilde w_c(a,b)}{\sum_c \sum_{a'} \tilde w_c(a',b)},$$

the probability that a user describes document term $b$ with query term $a$. Each device then personalizes with a shrinkage estimate

$$T_c(a \mid b) = \frac{n_c\,\hat T^{\text{local}}_c(a \mid b) + \lambda\,T(a \mid b)}{n_c + \lambda}.$$

Here $\hat T^{\text{local}}_c$ is the table computed from the client's own pairs and $\lambda$ controls how much to trust the pool. This is the standard empirical-Bayes way to borrow strength when local counts are small. The server adds high-$T$ query terms to documents before indexing. Devices use $T_c$ only for cheap query-side lookups.

The prediction that could fail is specific. Gains over public expansion should be large where user vocabulary differs from the corpus in community-specific ways, and near zero on generic web question answering. Gains should also grow with the number of clients even when every client holds only one to five pairs.

Relation to Prior Work

Two bodies of work motivate the idea without addressing it.

Federated recommenders fight local sparsity by giving sparse clients external structure. FedRKG samples public knowledge-graph subgraphs to anchor item embeddings. CoFedRec clusters users by category and adds contrastive learning, raising HR@10 on MovieLens-100K from 72.85 to 77.52. FedGDVE and DARD select useful public reference interactions with reinforcement learning or influence functions. All four operate on item or entity IDs, and FedRKG explicitly leaves rich textual overlap out of scope. We carry their central lesson, that sparse clients need pooled evidence and filtering against negative transfer, over to words.

On the lexical side, the relevant work supplies both methods and constraints. SPRINT shows that SPLADEv2's learned document expansion is what drives its BEIR advantage, reaching 0.470 average nDCG@10 against 0.429 for BM25. It also reports that neural sparse models run 3.5x to 46x slower than BM25. We keep expansion at index time, as SPLADE does, but learn it from pooled user feedback instead of a masked-language-model head trained on public data. RaFe trains query rewriters from reranker feedback, but it relies on a centralized pool of tens of thousands of examples. ECLASS-augmented product search constrains our design in two ways. It found that LLM query rewriting degraded industrial retrieval by dropping identifiers, which argues for additive expansion over rewriting. It also found that ontology metadata closes much of the gap, which makes ontologies a competitor and a possible complement.

BEIR documents that dense models trained on one corpus break under vocabulary shift. The Touché-2020 re-evaluation shows BM25 at 0.785 nDCG@10, still above DRAGON+ at 0.718. Together they warn that any method altering term matching must be checked for regressions.

The federated search systems RAGRoute and FedMosaic distribute documents across silos. They assume a shared embedding space and do not learn from user vocabulary, so they address a different problem.

Research Program

We will build the pipeline end to end. First, a client-side counter extracts term pairs from accepted results, using a public tokenizer and a stopword and IDF filter so that document boilerplate does not dominate. Second, secure summation with the Gaussian mechanism provides a stated privacy budget $\varepsilon$ per client. Third, the server-side expansion step appends the top-$k$ translated terms to each document, with weights capped so that document length normalization is preserved. Fourth, the on-device shrinkage personalization handles query-side lookups.

The analysis has three parts. The first is a sparsity-and-population sweep: pairs per client from 1 to 50, and client counts from 100 to 50,000. This maps where local learning fails and where pooling saturates.

The second is a study of the privacy–jargon tension. Rare community terms are both the most useful and the most identifying, so we measure how much of the gain survives as $\varepsilon$ shrinks. We also measure how much survives when terms must appear across at least $m$ clients before release.

The third is a heterogeneity study. Clients are grouped into communities with different jargon, and we test whether the single global table blurs them. If it does, we add one level of community clustering in the spirit of CoFedRec, applied to term-pair vectors instead of item embeddings.

Baselines

The naive reference point is plain BM25 on the unexpanded index. SPRINT reports 0.429 average nDCG@10 for it on BEIR, and the ECLASS study reports 31.4% Hit_Rate@5 on its expert queries. A second weak reference is local-only learning, in which each client uses only $\hat T^{\text{local}}_c$. This shows directly that one client's data cannot close the gap.

The strongest existing method for index-time lexical expansion is SPLADEv2 run zero-shot. It is the right competitor because it attacks the same failure with the same deployment shape and uses no private data. Its reported BEIR average is 0.470 nDCG@10 (SPRINT). We will also run docT5query, which BEIR reports beats BM25 on 11 of 18 datasets. Per-dataset SPLADEv2 numbers for our specific splits: no reported result found on Lacuna, so we will rerun it ourselves on identical corpora and query splits. Beating it means higher nDCG@10 on the jargon-heavy datasets at matched index size and BM25-level query latency. We also require no significant loss on generic datasets.

On the ECLASS catalog, the strongest system is the dense-plus-reranker "basicrr" pipeline, reported at 94.3% Hit_Rate@5 with 11.45 seconds mean latency (ECLASS). We do not expect to beat its accuracy. We will report how much of the gap from BM25 we close at millisecond latency. We will also test our table combined with ECLASS category augmentation.

The strongest federated competitor is FedAvg fine-tuning of SPLADE on the same client pairs under the same privacy budget. Here FedAvg means the standard federated recipe: clients fine-tune locally and the server averages model weights. We match rounds and give both methods the same hyperparameter-search budget.

The key ablations are the global table without personalization, pooling without noise, and a shuffled-pairs control. The shuffled control pairs queries with random documents and should erase the gain if the gain comes from real feedback. The centralized table built from pooled raw logs is the upper bound.

Why not just…?

Why not just expand documents with SPLADEv2 or docT5query and skip user data entirely? This is the real threat. On generic question answering, public expansion may capture nearly everything, and we expect our method to tie there. The claim rests on community-specific vocabulary, where public expansion models trained on web data are exactly the out-of-domain case BEIR warns about. The ECLASS trainee queries and the CQADupStack subforums test this directly. If SPLADEv2 matches us there too, the core claim is false, and we will say so.

Why not just federate a neural retriever with FedAvg? It is a legitimate alternative, and it may match accuracy. Its costs are on-device backpropagation and model-sized uploads every round. Evidence from sparse federated recommendation also suggests that uniform averaging over very small, skewed clients struggles: FREIB reports an MAE of 0.9371 for the best standard baselines, against 0.7369 for its method. Our head-to-head comparison at equal privacy budget settles accuracy. If FedAvg-SPLADE wins on accuracy, our contribution narrows to a far cheaper operating point, and we will report the accuracy-per-byte and accuracy-per-joule curves that make that trade explicit.

Why not just send queries to a cloud LLM for rewriting? Doing so ships private queries off the device, which is the thing the setting forbids. It is also slow: RaFe reports LLM judging at 78 hours for 30,000 instances versus 0.67 hours for reranker feedback. And rewriting has been observed to strip technical identifiers and hurt industrial search (ECLASS). We will still include a local small-LLM rewriter as a reference on the ECLASS queries.

Proposed Experiments

We simulate federations from benchmarks that have natural communities. CQADupStack, from BEIR, is split so that each StackExchange subforum is a community. Its questions are divided into many clients of one to fifty labeled pairs each, with held-out queries for evaluation. SciFact, FEVER, and Natural Questions use their training queries partitioned the same way. NQ is the generic control where we predict no gain over SPLADEv2. The ECLASS catalog of 10,346 components uses trainee and expert queries in cross-validation as a small, jargon-heavy stress test measured by Hit_Rate@5 and MRR. Touché-2020 serves as a regression check: expansion must not push results toward the short-passage failure, so BM25's 0.785 must not drop.

The primary metric is nDCG@10. Secondary measurements are query latency, index growth, upload bytes per client, and privacy budget.

Three results would be decisive. First, on CQADupStack and ECLASS, federated tables built from clients holding five or fewer pairs should beat both local-only learning and zero-shot SPLADEv2. They should close most of the distance to the centralized-table upper bound as the client count grows. Second, the shuffled-pairs control should return to BM25 levels. Third, on NQ the advantage over SPLADEv2 should vanish, confirming that the benefit comes from community vocabulary and not from extra supervision in general. If the gain survives at a privacy budget of $\varepsilon \le 8$ with a minimum-client threshold, the method is deployable. If it survives only without noise, the negative result is itself informative: it would show that the rare jargon needed to close the gap is the part that privacy protection removes.