Teaching Text-Only Retrievers the Vocabulary of a Document's Neighbors

When a searcher and an author describe the same idea in different words, keyword search returns nothing and dense retrievers often do little better. The question "how to fix a broken pipe" and the document "remediating industrial conduit fractures" share no words. In a natural search log, about a fifth of correct answers share no content word with the question (WikiQA). The text needed to close this gap often already exists, just not inside the target document. Papers that cite a study, pages that link to an entity, and later cases that rely on a precedent describe the target in other people's words, and those words are often closer to how a newcomer would phrase a query. Standard retrievers throw that information away because they score every document in isolation, as if the corpus had no links. This is the document independence assumption.

We propose to train a retriever that, given only a document's own text, predicts where that document would sit in query space if its graph neighborhood were visible. A graph-aware teacher reads each document together with its citing and linked neighbors. A text-only student encoder is distilled to reproduce the teacher's query–document scores. At search time the student needs no graph. It therefore works on new, uncited, or unlinked documents, which are exactly where graph methods break and where vocabulary mismatch hurts most. The claim we test is narrow: structure-distilled text encoders recover a large share of the graph's benefit on queries that share no words with their target, including documents with no links at all.

Core Claim

Existing hybrid text–graph methods use structure at query time or for closed-world classification. CG-RAG runs a graph transformer over a chunk-level citation graph, mixing BM25 and dense signals during message passing. Its authors note that its ceiling is the initial dense retriever: a lexically mismatched, weakly cited document that never enters the candidate subgraph cannot be rescued. G-Retriever has the same weakness, because its subgraphs start from frozen SentenceBERT nearest-neighbor search over isolated nodes. CaseLink bridges lexically different legal cases by routing through hand-curated statutory charge nodes, and that ontology does not exist in most domains.

The method we reuse comes from GLEM. GLEM alternates training between a language model and a graph neural network (a model that updates each node by aggregating its neighbors) through pseudo-label exchange. It shows the resulting standalone language model makes better predictions on unseen nodes without seeing edges. GLEM, however, is about node classification with a closed label set. It never handles open-ended queries, ranking, or the asymmetry between query wording and document wording.

Our contribution is to move the "structure distilled into text" idea into first-stage retrieval. We then measure it exactly where the theory says it should matter: queries with zero lexical overlap with the target, and documents with no edges. If it works, the student also fixes CG-RAG's entry-point problem, because a better first stage feeds better candidates to any graph re-ranker.

Research Program

The teacher is a dual-encoder retriever. Its document side follows GraphFormers: neighbor representations are fused into each transformer layer of the document encoder. GraphFormers uses one-directional attention from the center node to cached neighbors, which keeps the extra cost near 3.5%, and it masks part of the center text during training so the model cannot ignore its neighbors. We change two things. First, neighbors are chosen by edge type, with citing contexts preferred over cited ones, because citing text describes the target from the outside. Second, the teacher is trained on query–document relevance rather than link prediction.

The student is the same encoder with the graph module removed. It is trained on the teacher's score distributions over a shared candidate pool, including documents the teacher saw with neighbors. Let $s_T(q,d)$ and $s_S(q,d)$ be the teacher and student scores for query $q$ and document $d$, let $C_q$ be the candidate pool, and let $p_T$ and $p_S$ be the softmax of each score over $C_q$. The student minimizes

$$\mathcal{L}_S = \mathcal{L}_{\text{InfoNCE}}(q, d^+, C_q) + \lambda \, \mathrm{KL}\big(p_T(\cdot \mid q) \,\|\, p_S(\cdot \mid q)\big),$$

where $d^+$ is the gold document and $\lambda$ weights distillation against direct supervision. Following GLEM, we then let teacher and student exchange signals in alternating rounds. The student's scores become pseudo-relevance labels on unlabeled queries, and these are used to retrain the teacher. We stop when held-out recall stops improving, and report how many rounds help.

To keep queries from being only the labeled ones, we generate unlabeled training queries from citation contexts: the sentence in a citing paper that mentions the target. These sentences are naturally phrased in the citer's vocabulary rather than the author's.

Baselines

The naive reference is BM25 over document text alone. It fixes the floor and identifies the zero-overlap subset, where BM25 is near zero by construction.

The main text-only competitor is a dense bi-encoder fine-tuned on the same labeled queries with the same backbone, compute, and number of tuning trials as the student. This is the retriever CG-RAG and G-Retriever start from. On PubMedQA-1k with a citation graph, CG-RAG reports ColBERT at Hit@1 0.913 (CG-RAG). ColBERT is the strongest independent-document number in the file, and we will reproduce it.

The strongest graph-using method is CG-RAG's LeSeGR retriever. It reports Hit@1 0.961 on PubMedQA-1k and QA accuracy 0.835 on PapersWithCodeQA, against 0.769 for the best hybrid non-graph baseline. On COLIEE we compare against CaseLink, which reports NDCG@5 of 70.3% (2022) and 49.8% (2023). We do not expect a text-only student to beat these systems when full graphs are available. Beating them means two things: matching them within a small margin on fully linked documents, and exceeding them on unlinked documents, where they lose their structural input. Separately, we test whether the student, used as CG-RAG's first stage, raises CG-RAG above 0.961.

Two baselines isolate the key ingredient. In the first, the same student is distilled from a teacher without the graph module. This tests whether the gain comes from neighbors or merely from distillation and extra queries. In the second, the student is trained contrastively with links as positive pairs, a direct "use edges as supervision" alternative with no teacher.

Why not just…?

Why not just append neighbors' text, such as citing sentences or anchor text, to each document before indexing? This is the most serious threat, and it may match us on documents that already have links. It is cheaper and needs no distillation. Its limit is structural: it does nothing for a document with no incoming links at indexing time, such as a new paper, an uncited case, or an orphan enterprise page. That is the cold-start regime CaseLink identifies as hard and LinkBERT concedes for link-dependent methods. We include neighbor-text expansion as a full baseline. If it equals the student on linked documents, we report that plainly and position the contribution as the cold-start result alone.

Why not just generate pseudo-queries for every document with an LLM? Multi-view indexing with synthetic queries (MINDER) is text-only and so also handles cold start. It is the fair competitor on unlinked documents. Its risk is that the generator guesses vocabulary from the document's own wording, which is the vocabulary that already mismatches. Neighbors, by contrast, supply wording from other authors. The decisive comparison is on the zero-overlap, unlinked subset with matched index size. If pseudo-query expansion wins there, our hypothesis about where bridging vocabulary comes from is wrong.

Why not just rewrite the query instead of changing the documents? Rewriters trained with ranking feedback (RaFe, GRAPE) and retrieve-then-generate loops (InteR) attack the same gap from the query side. Their drawbacks are known. GRAPE documents rewriters gaming similarity rewards by appending generic terms. InteR finds that more than two rounds adds drift. Every query pays multiple LLM calls before lookup. They are also complementary to our approach, so we test them both as competitors and stacked on top of the student. If a GRAPE-style rewriter over the plain bi-encoder matches the student, the document-side approach is not worth its training cost.

Proposed Experiments

We use the graph-equipped benchmarks in the file. These are PubMedQA-1k with its 7,849-paper citation graph and PapersWithCodeQA with 12,171 papers (both from CG-RAG), COLIEE 2022 and 2023 with precedent links, and HotpotQA over Wikipedia hyperlinks as used by LinkBERT.

Every result is reported in four cells. Queries are split by whether they share any content word with the gold document, following the WikiQA overlap analysis. Gold documents are split by whether their edges were visible during training and indexing. To create unlinked documents we use two protocols: hiding all edges of a random 30% of documents, and a time split in which documents after a cutoff date contribute no edges. Metrics are Recall@50 and Hit@1 for retrieval, NDCG@5 on COLIEE to match CaseLink, and downstream QA accuracy on PapersWithCodeQA.

The decisive check is the zero-overlap, unlinked cell. The student should beat both the fine-tuned bi-encoder and pseudo-query expansion there. The no-graph-teacher ablation should lose most of that gain, which would show the gain comes from neighbor vocabulary rather than distillation alone. In the linked cells, the student should close a substantial fraction of the gap between ColBERT's 0.913 and LeSeGR's 0.961 Hit@1 without using the graph at query time.

A secondary check is whether the student, plugged in as CG-RAG's first-stage retriever, increases the fraction of gold documents that enter the message-passing subgraph, and whether that lifts end-to-end accuracy. On COLIEE, we ask how much of CaseLink's charge-bridge advantage the student keeps when charges are withheld at test time. This tells us whether learned neighbor vocabulary can stand in for a curated ontology.

Finally, we inspect the terms the student moves closest to each document. Bridging terms that come from citing contexts rather than generic high-frequency words would rule out the reward-gaming failure GRAPE describes. If the gains appear only in linked cells, the proposal collapses to neighbor-text expansion and we will say so.