Polygenic scores add up many small genetic effects into a single number that predicts disease risk. They work far worse in people whose ancestry differs from the mostly European cohorts they were trained on. When 166 published scores were re-run in UK Biobank, their accuracy relative to Europeans fell to 39.4% in African, 46.6% in East Asian and 51.3% in South Asian participants (Moreno-Grau et al.). Clinicians, guideline writers and methods developers mostly learn how large this "portability gap" is by reading papers, not by re-running models. Nobody has checked whether the portability that authors report matches what independent re-evaluation finds, or whether the literature's structure predicts the mismatch. By structure we mean who evaluated whose model, on which overlapping cohorts, and how often a claim gets cited.
We propose to build a portability ledger from the Web of Science Core Collection (WoSCC), the curated citation database that bibliometric studies use to map entire fields. A language model will read the full text of every indexed paper that evaluates a polygenic score across ancestries and pull out structured records: model, training ancestry mix, target ancestry, metric, value and cohort. We will join these records to independent re-evaluations of the same models and test a specific claim. The claim is that author-reported cross-ancestry accuracy is systematically higher than independently measured accuracy, and that the excess is concentrated where the WoSCC co-authorship and citation graphs show a model's own developers evaluated it, or the target cohort overlapped the training cohort. In this project the citation graph is the measuring instrument. Portability is what gets measured.
Core Claim
The quantity of interest for each model $m$ and target ancestry $a$ is the relative accuracy
$$RA_{m,a} = \frac{R^2_{m,a}}{R^2_{m,\mathrm{EUR}}},$$
where $R^2_{m,a}$ is the variance explained in ancestry group $a$ and $R^2_{m,\mathrm{EUR}}$ is the same quantity in a European test set. For binary diseases we use the analogous ratio with AUC minus 0.5 in place of $R^2$. We call $\Delta_{m,a} = RA^{\text{reported}}_{m,a} - RA^{\text{independent}}_{m,a}$ the overstatement.
We hypothesize three things. First, $\Delta$ is positive on average. Second, $\Delta$ is larger when the evaluating team shares authors with the developing team. Third, $\Delta$ is larger when the evaluation cohort shares participants or consortium membership with the discovery cohort, which is the uncorrected-overlap inflation documented by Park et al.. If $\Delta$ turns out to be near zero, the ledger instead becomes a validated, low-cost way to monitor the gap, which is still a useful result.
How This Differs From Prior Work
Two strands of prior work motivate the project, and neither does what we propose.
Bibliometric mapping on WoSCC, Scopus and PubMed describes where research is produced and how clusters form. Examples include institutional dominance in brain-injury research (Qi et al.), fragmented collaboration in carbon-neutrality research (Chen et al.), and US/UK concentration in nurse-staffing citation clusters (Morioka et al.). BIOLITMAP geolocates topic models of bioinformatics papers. All of these stop at metadata. They never read the numbers inside a paper, so they cannot say whether a field's quantitative claims hold up.
The genomics strand measures the gap directly, without asking how the literature reports it. Moreno-Grau et al. re-evaluated catalogued models. Liu et al. showed that bone-density prediction falls with genetic distance, down to $R^2 = -0.089$ in UK Biobank Africans. Fritsche et al. showed that ancestry shifts score distributions enough to invalidate shared risk thresholds.
We reuse methods from two further papers. Ingram et al. showed that small local language models can separate substantive contributions from incidental keyword mentions in abstracts. We extend this from abstracts to full-text numerical extraction. Giles et al. built entity graphs from acknowledgment sections and reported an extractor F1 of 91.9%. We use that idea to detect shared cohorts and funders between papers.
Existing methods for closing the gap operate on genotypes, not on the literature. These include X-Wing's local-correlation weighting (Miao et al.), GAUDI's local-ancestry fused lasso (Sun et al.), DisPred and PUMAS-ensemble. They serve here as the methods whose reported gains we audit. We do not compete with them.
Research Program
Corpus. We will query WoSCC for polygenic score and genome-wide association papers from 2010 onward that report any evaluation in a non-European or admixed group. Following standard bibliometric practice (Cui et al.), we restrict to research articles and exclude reviews and editorials. WoSCC is the backbone because it indexes statistical-genetics and computational venues that PubMed covers unevenly, and because its citation links are standardized across disciplines. Full texts and supplementary tables will be retrieved where licensing allows.
Extraction. A local instruction-tuned model will fill a fixed schema for every evaluation reported in a paper:
- the score's identifier, linked to its PGS Catalog ID where one exists;
- the training ancestry proportions;
- the target ancestry and cohort name;
- the linkage-disequilibrium reference panel, meaning the external genotype sample used to estimate correlations between nearby variants;
- the metric, the value, and the European comparator.
A second pass will check that each extracted number appears verbatim in the source text or table. This guards against the claim-entailment failures that Liebling et al. observed when generating text over OpenAlex. We will hand-annotate a gold set of about 300 papers to measure extraction accuracy. No reported result was found on Lacuna for language-model extraction of ancestry-stratified performance metrics from genomics full texts, so this gold set is itself a contribution.
Network features. From WoSCC we will derive four features for each extracted evaluation:
- author overlap between the evaluating paper and the model's origin paper;
- cohort overlap, inferred from named cohorts and from consortium and funder mentions in acknowledgment sections using Giles-style extraction with name normalization;
- whether the evaluation appears in the origin paper itself;
- citation counts accumulated within three years of publication.
Ground truth. The independent reference is the set of UK Biobank re-evaluations from Moreno-Grau et al., matched by model and ancestry. Where we have data access, we will add our own re-scoring of catalogued models in UK Biobank's African, East Asian and South Asian subsets. Our re-scoring will use the same covariates and the same $R^2$ definition, so every pair is compared on identical footing.
Propagation analysis. For each score that is used in a clinically oriented citing paper, the extraction model will label whether the citing paper states the training ancestry limitation. This lets us estimate whether caveats survive citation.
Baselines
The naive reference is keyword-level bibliometrics. It records portability from abstract sentences found with a Boolean query and applies no full-text reading and no network features. This is what a standard CiteSpace or VOSviewer study would produce. If our full-text ledger does not correlate better with independent re-evaluation than abstract-level numbers do, the extraction step adds nothing.
The strongest competitor is the PGS Catalog's curated performance records. These are the source Moreno-Grau et al. drew models from, and they are maintained by human curators. We compare both sources on the same set of models against the same independent re-evaluations. Two things count as beating the catalog: covering substantially more model–ancestry pairs at comparable agreement, or explaining variance in $\Delta$ that catalog fields cannot.
The strongest existing predictor of true portability is genetic distance alone. Liu et al. showed accuracy falls as distance between training and target populations grows. We fit a genetic-distance-only regression of $RA^{\text{independent}}$, then add literature features using the same cross-validation folds. The benchmark effect size is the 39.4% African relative accuracy from Moreno-Grau et al. Beating the baseline means cutting held-out prediction error of independent $RA$ below what genetic distance plus trait category achieves.
The key ablation removes the co-authorship and cohort-overlap features. This tests whether the WoSCC graph itself is what explains overstatement, as opposed to paper-level variables such as sample size or publication year.
Why not just…?
Why not just use the PGS Catalog? It is curated, and it may well be as accurate as our extraction wherever it has entries. That is a real threat to the extraction component, and the head-to-head comparison above settles it. The catalog has two limits that our design addresses. It records scores that were deposited, not every cross-ancestry evaluation in the literature, including evaluations of undeposited scores. It also stores no information about who evaluated whom or which cohorts overlap, so it cannot test the overstatement hypothesis at all.
Why not just re-run every model in a biobank, as Moreno-Grau et al. did? Independent re-evaluation is the gold standard, and we use it as ground truth. It is limited to the ancestry groups and sample sizes available in accessible biobanks, such as roughly 18,000 non-European UK Biobank participants. It also cannot re-evaluate models whose weights were never released. It says nothing about how the literature a clinician actually reads is biased. The ledger is meant to be calibrated against re-evaluation, not to replace it.
Why not just model portability from genetic distance? Liu et al. make this a strong baseline, and it may explain most variation in true portability. Our claim is about a different quantity. We are trying to explain the gap between reported and true portability, which genetic distance should not predict. If literature features add nothing beyond genetic distance in predicting $\Delta$, the overstatement hypothesis fails.
Why not just use PubMed? PubMed is open, but Morioka et al. note that it lacks cross-disciplinary citation indexing. Many polygenic methods appear in statistics and computing venues. We will report how many ledger entries PubMed alone would miss.
Proposed Experiments and Decisive Checks
Extraction accuracy. Measure precision and recall for each schema field on the gold set. Compare against a regular-expression extractor in the style of Giles et al..
Primary test. Estimate the mean $\Delta$ over matched model–ancestry pairs, with a bootstrap confidence interval. Regress $\Delta$ on author overlap, cohort overlap, and whether the evaluation appears in the origin paper, controlling for trait, target sample size and genetic distance.
Negative control. Run the same regression on European-to-European external validations, where no ancestry shift exists. If author overlap predicts overstatement equally there, the effect reflects self-evaluation in general, not portability reporting.
Method-claim audit. For papers reporting gains from multi-ancestry methods, including the 14.1%–119.1% relative $R^2$ gains of X-Wing (Miao et al.) and GAUDI's 63.8% gain over PRS-CSx (Sun et al.), test whether gains reported by developers exceed the gains later reported by teams with no author overlap in WoSCC.
Main risk. The number of matched pairs may be small. We will report the achieved sample size and minimum detectable effect before examining $\Delta$.
References
- Polygenic risk score portability for common diseases across genetically diverse populations
- Overestimated prediction using polygenic prediction derived from summary statistics
- Bibliometric Analysis of Chronic Traumatic Encephalopathy Research from 1999 to 2019
- Climate Change, Carbon Peaks, and Carbon Neutralization: A Bibliometric Study from 2006 to 2023
- Citation Network Analysis of Nurse Staffing Research from the Past Two Decades: 2000–2022
- BIOLITMAP: a web-based geolocated, temporal and thematic visualization of the evolution of bioinformatics publications
- Variability in performance of genetic-enhanced DXA-BMD prediction models across diverse ethnic and geographic populations: A risk prediction study
- On cross-ancestry cancer polygenic risk scores
- Agentic AI for Improving Precision in Identifying Contributions to Sustainable Development Goals
- Towards Building and Analyzing a Social Network of Acknowledgments in Scientific and Academic Documents
- Quantifying portable genetic effects and improving cross-ancestry genetic prediction with GWAS summary statistics
- Improving polygenic risk prediction in admixed populations by explicitly modeling ancestral-differential effects via GAUDI
- Improving genetic risk prediction across diverse population by disentangling ancestry representations
- Optimizing and benchmarking polygenic risk scores with GWAS summary statistics
- Top 100 most-cited articles on renal cell carcinoma: A bibliometric analysis
- Towards AI-assisted Academic Writing