Letting Noisy Lay Mentions Speak for Medical Concepts

Patients, clinicians and ontologies name the same things differently. A Reddit post says "my stomach burns after I eat", a clinical note says "GERD sx", and the ontology entry says "gastro-oesophageal reflux disease". Retrieval systems that link text to ontology concepts fail most often when the query shares no words with any of the concept's official names. We propose to close that gap on the concept side of the index. Each concept's representation would absorb the everyday language that surrounds it in unlabeled text, collected automatically by string-matching the ontology against a raw corpus. That matching is distant supervision, meaning labels produced by aligning text with an existing database rather than by human annotators.

The catch is that automatically collected mentions are noisy. Polysemous strings like "cold" land on the wrong concept, and many surrounding contexts say nothing useful. Our central bet is that treating the ontology plus the mined mentions as a heterogeneous graph handles this noise. Such a graph has typed nodes (concepts, official synonyms, mined mention contexts) and typed edges (is-a, synonym-of, mentioned-in). A graph encoder that learns a separate gate per edge type can decide which mention contexts to trust, and it can learn that from the same distant signal. Because all of this happens on the concept side, it is computed once, offline. At query time the system is an ordinary two-tower retriever: a query encoder, cached concept vectors, and nearest-neighbour search.

Core Claim

In zero-shot concept retrieval, much of the lexical gap is a coverage problem: concept-side text lacks lay vocabulary. Distantly aligned corpus mentions contain that vocabulary but also contain noise that grows with coverage. We claim that a relation-typed, gated aggregation over a heterogeneous concept graph uses these mentions better than either alternative. It should beat ignoring them, which is the current best annotation-free approach, and it should beat flattening them into the concept's text. The gains should concentrate on queries with zero word overlap with any official synonym.

The closest work is Zero-shot Medical Entity Retrieval without Annotation. It mines positive pairs from ICD-10, SNOMED CT and UMLS synonyms and is-a links, then trains a Siamese encoder with in-batch negatives. It reaches Recall@1 of 0.32 on MedMention, against 0.04 for BM25. That paper motivates this proposal and supplies part of the method. It shows that ontology structure is a usable training signal. It never looks at unlabeled corpus text, though, and it uses structure only to create training pairs. No message passing happens when concepts are indexed. Our proposal differs on both counts. Structure and corpus mentions enter the concept representation itself, and the noise problem that arises is handled by the typed gates.

How the Pieces Fit

The graph has three node types. Concept nodes carry the preferred name and definition. Surface-form nodes carry synonyms and "see also" strings. Mention nodes each hold a roughly 64-token window of unlabeled text in which some surface form of the concept was string-matched. Edges are typed as is-a, synonym-of and mentioned-in. The node types themselves come from ontology releases, so no schema needs to be designed by hand. That avoids the schema-engineering cost that usually burdens heterogeneous graph models.

The approach draws on three existing ideas. First, the mention collection follows Jointly Embedding Entities and Text with Distant Supervision. That paper deliberately updates every entity a polysemous string maps to and relies on billions of tokens to wash out the errors. We take its unannotated dictionary alignment but replace "wash it out with scale" with a learned per-mention gate, because we want deep contextual encoders rather than Skip-gram vectors. Second, relation-specific attention comes from the Heterogeneous Graph Transformer and Heterogeneous Graph Attention Network. Third, the decision to keep only the language model at inference time follows GLEM. GLEM alternates training a language model and a graph network on each other's pseudo-labels and shows that the language model alone can predict on unseen nodes without graph edges. GLEM, however, works on homogeneous citation graphs for node classification. We move its idea of structure-aware training with cheap inference to typed graphs and to cross-text retrieval.

The concept vector for concept $c$ is

$$ z_c = f(t_c) + \sum_{r \in R} \sum_{u \in N_r(c)} g_r(t_c, t_u)\, W_r f(t_u), $$

where $f$ is a shared transformer text encoder, $t_c$ and $t_u$ are node texts, $R$ is the set of edge types, $N_r(c)$ are the neighbours of $c$ under edge type $r$, $W_r$ is a type-specific projection, and $g_r \in [0,1]$ is a learned gate. The gate is normalised within each edge type, so the many mention nodes of a common concept cannot drown out its few synonym nodes. This addresses the type-dominance problem noted for heterogeneous graph networks.

Training uses two distant sources of pseudo-queries, scored with a contrastive loss against $z_c$. The first source is the synonym and is-a pairs of Kong et al. The second is masked mention contexts. We take a mined window, delete the matched surface form, and ask the model to retrieve the concept from what remains. The remaining context is exactly the lay paraphrase that real queries contain, which forces the model to learn the lexical gap without annotation. A mention used as a query is removed from its concept's neighbourhood for that step, so the model cannot copy text.

Training then alternates, in GLEM style. Once the gated graph model scores each mined alignment, low-scoring mentions are down-weighted as pseudo-queries in the next round. Mention nodes whose gates go to zero are dropped from the graph. This is our denoising step, and it replaces hand-written constraints of the kind explored in Differentiable Logic Programming for Distant Supervision.

Research Program

We will build the mention graph for UMLS, SNOMED CT and ICD-10. The raw corpora are PubMed abstracts, a public health-forum or Reddit dump, and, if access allows, de-identified clinical text. We will deduplicate the corpora against every evaluation set at the document level, and report the overlap statistics we find, because leakage from COMETA's Reddit source is the most likely way this study could fool itself.

The analysis asks three questions. Does the typed, gated graph beat flattening at an equal text budget? Where do the gains come from, measured by stratifying queries by word overlap with official synonyms? How robust are the gates to injected alignment noise? We will also test inductive transfer to held-out concepts. These concepts have no mined mentions of their own and receive signal only through is-a neighbours, which is a setting where a pure text encoder has nothing extra to use.

Baselines

The naive reference point is BM25 over the concept's official names. Kong et al. report MedMention Recall@1 of 0.04 for it, and we will rerun it on our exact concept inventory and splits.

The strongest existing annotation-free method is the Kong et al. "Comb" Siamese encoder, at MedMention Recall@1 of 0.32 as reported. We will re-implement it with the same backbone encoder, the same ontology release and the same number of training steps and hyperparameter trials as our model. This is the number to beat. Beating it means a statistically significant Recall@1 gain on MedMention and COMETA under a paired bootstrap, with most of the gain landing on zero-overlap queries.

The second strong competitor is the flattened-evidence bi-encoder. It uses the same mined mentions and the same masked-mention training, but concatenates the top-$k$ mentions (ranked by BM25 similarity to the concept name) into the concept text, as ECLASS-Augmented Semantic Product Search does with its category tree. We match its total concept-side token budget to the graph model's. The graph must beat this baseline, or the structural claim fails.

A neural sparse expansion retriever, SPLADEv2, will be fine-tuned on the same pseudo-queries. SPLADEv2 averages 0.470 nDCG@10 on BEIR according to SPRINT. We found no reported result on Lacuna for SPLADEv2 on MedMention or COMETA.

A frozen-SentenceBERT retriever over the same graph, as in G-Retriever's retrieval stage, tests whether learning matters at all.

Four ablations isolate the key ingredient. The first removes the gates, giving uniform averaging over neighbours. The second collapses all edge types into one, giving a homogeneous graph. The third keeps mentions only as training queries and never as graph nodes. The fourth drops the alternating denoising rounds.

Why not just…?

Why not just flatten the mentions into each concept's text? This is the most serious threat, because a long-context encoder might learn implicit gating by itself. Our view is that flattening forces a fixed selection before encoding, and noisy mentions then compete with synonyms for the token budget. The equal-budget comparison settles it. The noise-injection experiment adds wrong alignments at 10–50% and tracks how each method degrades. If flattening matches the graph at every noise level, we will report that the graph structure is unnecessary and that the contribution reduces to masked-mention distant supervision.

Why not just train Kong et al.'s encoder on more mined pairs, using the masked mentions as extra training queries? This is a genuine alternative and is exactly the third ablation. If most of the gain comes from better training queries rather than from index-time evidence, the ablation will show it. The prediction that separates the two explanations is the held-out-concept test: training-only mentions cannot help concepts that were never mentioned, but graph neighbours can.

Why not just rewrite the query with an LLM into ontology language? ECLASS-Augmented Semantic Product Search found that LLM rewriting degraded retrieval in several configurations by stripping technical nuance. RaFe reports only 2–3% downstream QA gains from trained rewriting. Rewriting also adds a generation call per query. We will still run an 8B-parameter rewriter in front of both the Kong baseline and our model, both to check whether its gains are orthogonal to ours and to measure its cost.

Why not just rerank with a cross-encoder? Reranking only repairs the ordering of candidates that first-stage retrieval already found. On zero-overlap queries, the correct concept is often not in the candidate set at all. We will report Recall@100 of each first stage, then apply the same reranker on top of every system.

Proposed Experiments and Decisive Checks

The primary benchmarks are MedMention (PubMed abstracts to UMLS) and COMETA (Reddit to SNOMED CT). We add 3DNotes if access permits, all as used by Kong et al.. Metrics are Recall@1, Recall@10 and Recall@100, reported overall and within query strata defined by maximum word overlap with any official synonym (zero, partial, exact).

A secondary transfer test builds the same kind of graph from DBpedia types and Wikipedia anchor contexts and evaluates on BEIR's DBPedia-Entity task with nDCG@10, following BEIR. No DBPedia-Entity-specific number appears in the file, so all baselines there will be rerun ourselves.

Three checks decide the claim. First, on zero-overlap MedMention and COMETA queries, the gated graph beats the flattened baseline at matched token budget. Second, on held-out concepts, the graph beats the training-only-mentions ablation. Third, under injected alignment noise, the gated model loses markedly less Recall@1 than both the ungated graph and the flattened baseline, and its learned gates rank injected wrong mentions below real ones, measured by AUROC.

If the first check fails but the third passes, the contribution narrows to a denoising method for distantly aligned mention text. If all three fail, index-time structure is not what closes the lexical gap here, and we will say so.