Most genetic studies of cause and effect are run on people of European descent. These studies use Mendelian randomization (MR), a method that treats inherited DNA variants as natural experiments to estimate whether an exposure such as BMI causes a disease such as type-2 diabetes. A finding from a European cohort often does not carry over to an African or East Asian one. The variants used as stand-ins may no longer track the causal mutation in that population, or they may act through other biological pathways. Literature search tools ignore this. Ask a hybrid search engine (one that combines keyword matching with neural semantic matching) whether LDL cholesterol protects against type-2 diabetes in East Asians, and it will return the strongest-matching European result with nothing marking the population mismatch.
We propose to treat "which population does this specific estimate come from" as a disambiguation problem over each claim's surrounding text, and to build it into the retrieval index itself. The key observation is that population scope in MR papers follows a default-plus-override pattern. The methods section sets a document-wide default ("all summary statistics are from European-ancestry individuals"). A sentence, a table row, or a replication paragraph then locally overrides it ("in Biobank Japan the association was null"). Because two-sample MR draws the exposure and the outcome from different datasets, the scope of a claim is really a pair: exposure population and outcome population. Document-level ancestry tags get the overrides wrong, and passage-level chunks lose the default. We will index atomic causal claims with a resolved population-scope pair, score them with a population-compatibility term on top of sparse and dense matching, and measure how often current systems present evidence from the wrong population as an answer.
Core Claim
Population scope is mostly recoverable from a claim's local context plus a single inherited document default. Making scope an explicit, retrieval-time field cuts "scope errors" by a large margin at a negligible cost in topical relevance. A scope error is a top-ranked claim offered as evidence for population P that was actually estimated in another population. Two cheaper alternatives fall short: document-level ancestry filtering, and appending the population to the query.
Where This Sits in Prior Work
Our scope resolver borrows from classic word-sense work, which showed that immediate neighbors often carry the decisive signal. Lexical Disambiguation: Sources of Information and their Statistical Realization resolves senses from syntactic neighbors and short co-occurrence windows. A Decision Tree of Bigrams is an Accurate Predictor of Word Sense shows that two-word features beat heavily engineered systems on SENSEVAL. Both supply the method. The bigram paper also constrains it: Pedersen notes that purely local cues fail when the decisive word is far away. That is exactly the case of a scope declared once in the methods section, and it is why we add an inherited default rather than relying on locality alone. Entity Commonsense Representation for Neural Abstractive Summarization and Disambiguating Noun Compounds motivate the claim-level unit. Both find that document-wide context adds noise that hurts fine-grained resolution.
The retrieval backbone reuses established hybrid designs:
- Sparse, Dense, and Attentional Representations for Text Retrieval interpolates BM25 with multi-vector dense scores. It also shows single dense vectors lose fidelity on longer text.
- Hybrid First-stage Retrieval Models for Biomedical Literature concatenates sparse and dense vectors for PubMed. It also shows pure dense retrieval collapsing on biomedical terminology (Batch 2 MAP 0.1173 versus BM25 at 0.2718).
- Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index indexes short phrases with dual representations.
None of these systems represent who a finding applies to. All three rank by topical relevance alone.
The nearest system is AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis. It indexes atomic chemistry claims with provenance under hybrid retrieval and builds a support/contradict evidence graph. We take its claim-store idea but add what chemistry does not need: a population-scope field resolved from context, and a scoring term that uses it. Contradictions between populations become a first-class output, so they are not averaged away.
On the genetics side, Causal associations between risk factors and common diseases inferred from GWAS summary data explicitly flags its European-centric data as a limitation, and Precision diagnosis for monogenic diabetes requires ethnicity specific criteria for genetic testing shows that White-derived criteria misclassify other groups. These papers motivate the problem. Leveraging the local genetic structure for trans-ancestry association mapping (LOG-TRAM) is the conceptual cousin. It borrows European information into Biobank Japan only in chromosome regions where local genetic structure is conserved. We do not reimplement it. We use it as an optional downstream signal (described below), and our contribution remains on the text side, which it does not touch. MRdb supplies structured MR results but no text retrieval. We use it as a source of distant supervision.
Research Program
Benchmark. No benchmark exists for population-scoped causal retrieval, so the first artifact is one. We will collect several thousand MR papers from PubMed of the kind represented in the file, from drug-target, microbiome, and metabolite studies to cross-trait work. Annotators will extract each reported causal estimate as a claim with these fields:
- exposure and outcome;
- direction and significance;
- instrument identifiers when given;
- the exposure-population and outcome-population pair.
The pair is labeled at the level of the specific estimate. Each label is marked as stated locally, inherited from a document default, or unrecoverable.
To scale labels cheaply, we add distant supervision. When a claim names a source dataset that can be matched to a dataset in the GWAS collections MRdb is built from (FinnGen, IEU Open GWAS), that dataset's documented population serves as a silver label. A hand-checked gold subset is used for evaluation.
Queries are written by people with an MR background, in the form "does X cause Y in population P." We oversample non-European P and include queries where the honest answer is "only European evidence exists."
Scope resolver. For each claim, a lightweight resolver predicts the population pair. It uses features from a small window around the estimate: cohort names, consortium names, biobank names, and ancestry descriptors such as "Han Chinese," "BBJ," and "East Asian," which need normalizing to one another. It also uses the document default extracted once from the methods and abstract. A learned gate decides whether a local mention overrides the default. This is the specific mechanism we test, because it addresses both of the failure modes noted above.
Retrieval. Each claim is indexed with a sparse vector, which keeps exact matches on rsIDs, gene names, and cohort names, and a dense vector for exposure, outcome, and paraphrase matching. Retrieval scores a claim $c$ for a query $q$ asking about population $P_q$ as
$$s(q,c) = \alpha\, s_{\text{sparse}}(q,c) + \beta\, s_{\text{dense}}(q,c) + \gamma\, \kappa(P_q, \hat{P}_c),$$
where $\hat{P}_c$ is the resolved exposure/outcome population pair. The compatibility term $\kappa$ is 1 when both members match $P_q$, an intermediate value when only one does, and lower when neither does. The weights $\alpha,\beta,\gamma$ are tuned on a development split. We score rather than hard-filter so that mismatched evidence still appears, labeled as such, when no matched evidence exists. The output groups results into "matched population," "other population," and "mixed exposure/outcome populations." It surfaces disagreements between groups explicitly.
Optional genetics signal. For European-only claims returned to East Asian queries, we test whether their named instruments fall in regions where LOG-TRAM-style local cross-population genetic correlation is low. If so, a stronger warning is justified. This analysis is secondary. The proposal stands without it.
Baselines
The naive reference is BM25 over whole abstracts with the population words simply present in the query. This is what a researcher typing into PubMed effectively gets. It sets the scope-error rate we expect to be high.
The strongest existing topical system is the unified sparse-dense first-stage retriever of Hybrid First-stage Retrieval Models for Biomedical Literature. Its HybridRerank variant reports an average MAP of 0.4056 across BioASQ 8 batches. That is the reference number we must not lose on topical relevance. We run it on our corpus with the same encoder size, the same training data, and the same tuning budget as our system. We give it the population in the query, which is the fairest version of "just let hybrid retrieval handle it." Beating it means a significantly lower scope-error rate at rank 10 with no more than a small drop in scope-blind nDCG@10, and we also report its BioASQ MAP to show the backbone is intact.
The claim-store competitor is an AskChem-style index (AskChem): the same claims, the same hybrid fusion, with an LLM reranker given the population. This represents the strongest current design pattern. We give it the same candidate pool and an equal rerank budget.
The decisive ablations isolate the scope mechanism, and each removes one ingredient:
- Document-level ancestry tags, with no local override.
- Local-window-only resolution, with no inherited default.
- Claim chunks with no explicit scope field, where the dense encoder must infer population implicitly.
- $\gamma=0$, which removes the compatibility term from scoring.
As an upper reference, a large LLM reads the full paper and assigns scope to each claim at indexing time. This tells us whether our cheap resolver leaves accuracy on the table.
Why not just…?
Why not just filter by document-level ancestry metadata? Many MR papers contain several populations: European discovery, East Asian replication, multi-ancestry meta-analysis. Under this pattern, a document tag is wrong for some claims in almost every multi-cohort paper. The document-tag ablation measures this directly. We will report the fraction of gold claims whose scope differs from their paper's majority scope. If that fraction is tiny in practice, the claim-level machinery is unnecessary, and we will say so.
Why not just append the population to the query and trust hybrid retrieval? Population words are a minor share of a claim's text. Topical match on exposure and outcome will dominate both the sparse and dense scores. Dense encoders also blur distinctions between rare identifiers, as Sparse, Dense, and Attentional Representations for Text Retrieval and Hybrid First-stage Retrieval Models for Biomedical Literature argue for terms like "JASPAR 2010" versus "JASPAR 2016." More importantly, a passage that inherits its scope from the methods section contains no population word at all. The strong hybrid baseline is run in exactly this mode, so the experiment settles the question.
Why not just rerank with an LLM told the population? This is the real threat. At query time, the reranker sees the claim's passage, not the methods section. It should therefore fail on inherited-scope claims, and it will cost far more per query, given the latency gaps between pipelines and single-stage systems reported by Contextualized Query Embeddings for Conversational Search. But if the LLM is instead given the whole paper at indexing time, it may match or beat our resolver. That is why it is our upper reference rather than a straw man. If it wins clearly, the resolver should be that LLM. The contribution then shifts to the scoped index, the scoring term, and the benchmark, which remain useful regardless of how scope is extracted.
Proposed Experiments
The primary evaluation is on the new benchmark. No reported result was found on Lacuna for any system on population-scoped causal retrieval.
We report scope-aware nDCG@10, which counts a claim as relevant only if it is on topic and in the queried population. We also report the scope-error rate at rank 10 and scope-blind nDCG@10. Results are stratified by queried population and by whether the gold scope was stated locally or inherited. For the resolver, we report pair-level accuracy, also split by local versus inherited. For "only European evidence exists" queries, we measure whether the system correctly reports the absence of matched-population evidence.
As a sanity check that the backbone is not degraded, we run the topical retriever on BioASQ 8 against the reported 0.4056 MAP.
The decisive check is the interaction between query population and scope source. If our gains concentrate on non-European queries and on inherited-scope or overridden claims, while the document-tag and local-only ablations each fail on one of those two subsets, the default-plus-override account is supported. If the population-in-query hybrid baseline already achieves a low scope-error rate, the problem is not where we think it is. The idea should then shrink to a benchmark-and-measurement paper.
References
- Lexical Disambiguation: Sources of Information and their Statistical Realization
- A Decision Tree of Bigrams is an Accurate Predictor of Word Sense
- Entity Commonsense Representation for Neural Abstractive Summarization
- Disambiguating Noun Compounds
- Sparse, Dense, and Attentional Representations for Text Retrieval
- Hybrid First-stage Retrieval Models for Biomedical Literature
- Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index
- AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis
- Causal associations between risk factors and common diseases inferred from GWAS summary data
- Precision diagnosis for monogenic diabetes requires ethnicity specific criteria for genetic testing
- Leveraging the local genetic structure for trans-ancestry association mapping
- MRdb: a comprehensive database of univariate and multivariate Mendelian randomization with large-scale GWAS summary data
- Contextualized Query Embeddings for Conversational Search