Only about one in five experimentally documented links between a non-coding RNA and a disease is causal. Here "causal" means that overexpressing or silencing the RNA changes the disease, as opposed to the RNA's level merely shifting alongside it. Current predictors of causal links are graph methods: they spread known causal labels through similarity networks. They therefore say nothing about RNAs or diseases that have no edges yet, and those are exactly the cases where a prediction would guide a new experiment. Language models trained on biomedical literature can produce a vector for any named entity. The obvious objection is that such vectors are built from co-occurrence, and co-occurrence is precisely what causal and merely correlated RNAs share.
This proposal turns that objection into the method. The curated databases already contain thousands of RNA–disease pairs that are documented as associated but judged non-causal. These pairs act as a natural control group: they are equally present in the literature, but causally different. We train a small scoring model on top of frozen domain text embeddings. It is required to rank causal pairs above these associated-but-non-causal pairs, and the part of the embedding that merely predicts "these two are associated" is removed. The result is a causal scorer that works from text alone. Its first real test is entities that the graph methods cannot score at all.
Core Claim
Domain-specific text embeddings contain usable signal about causal status only after the component that predicts general association has been removed. With that component projected out, a text-only scorer trained with non-causal associations as hard negatives should (i) match or approach graph-based causal predictors on entities already in the graph and (ii) produce useful causal rankings for RNAs and diseases held out of the graph entirely. Graph methods cannot operate in setting (ii). If the claim is right, the association-removal step should matter most on the causal-versus-non-causal task and least on causal-versus-unrelated. That asymmetry is the signature we test for.
Where This Sits
The causal ground truth comes from curation work. A Computational Model to Predict the Causal miRNAs for Diseases filtered HMDD v3.1 down to 6,667 causal microRNA–disease associations, based on gain- and loss-of-function evidence. Annotation and curation of the causality information in LncRNADisease found that 2,297 of 10,564 lncRNA–disease and 198 of 1,004 circRNA–disease associations are causal. These papers supply our labels, and above all they supply the non-causal associated pairs that serve as controls.
The predictive line (MDCAP; LE-MDCAP, which adds sequence edit-distance priors; DNI-MDCAP, which fills in missing network edges with Node2Vec random-walk embeddings before label propagation) motivates the proposal. Both later papers state that they cannot score entities lacking network edges. DNI-MDCAP's imputation is purely topological, so heavily studied entities get dense, self-reinforcing neighborhoods. We do not use propagation. We score pairs from text and use the graph methods as competitors and, later, as a complementary input.
Two negative results constrain the design. Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings found that initializing knowledge-graph embeddings with Clinical BERT vectors gave no gain on SNOMED-CT link prediction. So we do not expect raw language vectors to carry relational structure for free. We do not use text as an initialization. Instead, we train a supervised map from text space to a causal scoring space, and the causal labels shape the map directly. Edge, which lifts SNOMED link-prediction AUC from 0.773 to 0.916 by aligning a graph with an external-text graph, suggests that text helps when it is aligned under an explicit objective. EDGE models undifferentiated relatedness, however, which is exactly the signal we want to strip out. The biomedical language-model survey Pre-trained Language Models in Biomedical Domain: A Systematic Survey names the inability to separate cause from correlation as an open problem, and that inability is what we measure.
The method borrows from two general-domain papers. Knowledge Graph Embeddings for Causal Relation Prediction showed on Wikidata events that link predictors confuse similarity with causality unless non-causal relations are used as hard negatives, and that closed-world scoring penalizes true but unrecorded causal edges. We adopt both lessons in a molecular setting. CausE splits embeddings into causal and confounder parts using synthetic noise on FB15K-237 and WN18RR. We pursue a simpler and more checkable version of the same intuition: an explicit association component, estimated from real labels, is projected out.
Research Program
Each RNA and each disease receives a text embedding from a frozen biomedical encoder (PubMedBERT or BioLinkBERT, as described in the survey). The embedding is built by averaging the encoder's representation of the entity mention over PubMed abstracts published before a cutoff year $T$. RNA names follow miRBase and LncRNADisease identifiers. Diseases additionally get their MeSH definition text.
First, we fit a linear association probe $a$ that separates documented associated pairs (causal or not) from random unrelated pairs. Its weight vectors span a subspace $A$. We then project every embedding onto the orthogonal complement, $\tilde e = (I - P_A)e$. The causal scorer is a bilinear head,
$$s(r,d) = \sigma\big(f(\tilde e_r)^\top W g(\tilde e_d)\big),$$
where $f, g$ are small multilayer networks, $W$ is a learned matrix, and $\sigma$ is the logistic function. It is trained with a weighted cross-entropy loss. Causal pairs are positives. Associated-but-non-causal pairs are full-weight negatives. Unrelated pairs are down-weighted negatives, because some of them are unrecorded causal links, the closed-world problem flagged by the Wikidata causal-KGE study. The hypothesis lives in the projection step: removing $A$ should cost little on causal-versus-unrelated and help on causal-versus-non-causal.
Once the text-only scorer is characterized, we fuse it with DNI-MDCAP's network score via a learned gate that depends on node degree. This tests whether text adds anything on well-connected entities or only rescues cold ones.
Leakage is the central experimental risk. An abstract that reports a knockdown experiment for the target pair would let the encoder "read" the label. To prevent this, all splits are temporal. Labels come from associations whose supporting publication appears after $T$ (using the reference recorded with each database entry), and embeddings use only abstracts from before $T$. We additionally run a strict variant that drops every pre-$T$ abstract mentioning both members of a test pair.
Baselines
The naive reference points are two scores that require no learning. The first is literature popularity: the product of each entity's pre-$T$ abstract counts. The second is the raw cosine similarity between the unprojected text embeddings. If our scorer does not beat both on causal-versus-non-causal AUROC, the learned head has captured nothing beyond co-mention. We expect popularity to do well on causal-versus-unrelated and poorly on causal-versus-non-causal. Failing to observe that pattern would itself be informative.
The strongest existing method is DNI-MDCAP. It reports AUROC 0.870 on distinguishing causal from non-causal miRNA–disease associations on HMDD v3.2, 0.896 under ten-fold cross-validation and 0.889 on its independent test for causal prediction, and 0.821 when imputation is removed. It is the strongest competitor because it is the only model that explicitly targets incompleteness in the causal graph. We reimplement it on our temporal splits with the same similarity inputs and tune it with the same hyperparameter search budget as our model. We report both our rerun and the published figures, because the published splits are not temporal and the numbers may not transfer. LE-MDCAP, which reports causal-versus-non-causal AUROC 0.820 (MDCAP 0.695 in the same comparison), is run the same way as a second graph baseline. On in-graph entities, "beating" DNI-MDCAP means that the fused model exceeds the rerun DNI-MDCAP by a margin outside the across-seed confidence interval. Parity for the text-only model would already count as a positive result. On held-out entities, DNI-MDCAP cannot produce scores. The comparison there is against the naive references, and success means a causal-versus-non-causal AUROC clearly above both. There is no reported result on Lacuna for cold-start causal RNA–disease prediction.
A multi-relational factorization baseline, ComplEx trained with the recipe from Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? (HITS@10 0.793 on BioKG), is trained on the RNA–disease graph with "causal" and "non-causal-associated" as separate relation types. It gets the same tuning budget and tests whether relation typing alone captures the distinction.
Three ablations isolate the key ingredients. The first removes the projection (train on $e$ rather than $\tilde e$). The second removes hard negatives (treat all non-causal pairs as ordinary negatives). The third replaces the domain encoder with a general-domain BERT, testing whether domain adaptation matters for this task.
Why not just…?
Why not just add a PubMedBERT similarity network as one more input to DNI-MDCAP? This is the cheapest integration, and it is a real competitor on in-graph entities. We run it. It cannot help on entities with no edges, because propagation needs a seed neighborhood. It also feeds text similarity, which is dominated by association, into a method that already rewards dense neighborhoods. Chang et al.'s null result for Clinical BERT initialization suggests that unsupervised text similarity adds little to a relational model. If this variant matches our fused model on in-graph entities, we will say so and narrow the claim to the cold-start regime.
Why not just fine-tune a biomedical language model on abstracts mentioning each pair? A cross-encoder that reads pair-specific text is essentially relation extraction. On random splits it would likely win, because the abstracts describe the very experiments that define the label. It is the most serious threat, and it can only be judged under the temporal split and the strict variant that removes co-mentioning abstracts. We run it under both. If it still wins without co-mention text, the projection idea is unnecessary and we would adopt the cross-encoder instead.
Why not just use sequence similarity, as LE-MDCAP does? Edit distance on miRNA sequences reached 0.820 causal-versus-non-causal AUROC. However, it describes only the RNA side, cannot represent a new disease, and does not extend to lncRNAs, whose sequences are long and weakly conserved. We include sequence features as an extra input in the fused model. We report the lncRNA transfer results separately, because sequence priors have no obvious analogue there.
Why not just prompt a large language model? Graph-constrained Reasoning reports faithfulness of roughly 48–62% once graph constraints are removed, and the survey identifies causality as a known weakness of biomedical language models. We treat prompting as a reference point on a sampled subset of pairs, using the same cutoff-year framing, and measure its causal-versus-non-causal AUROC from elicited probabilities.
Proposed Experiments and Decisive Checks
The primary benchmark is the HMDD v3.2 causal subset (4,228 causal associations, 535 miRNAs, 302 diseases), with its non-causal associated pairs as hard negatives. The second benchmark is LncRNADisease v2.0 with the causality annotations above. Evaluation uses three regimes. In the first, test pairs involve entities seen in training. In the second, 20% of miRNAs are removed from the graph entirely. In the third, 20% of diseases are removed. We also run a transfer test: train on miRNA, then score lncRNA–disease causality for lncRNAs that were never seen. The metrics are AUROC and precision at top-15 for both causal-versus-non-causal and causal-versus-unrelated, echoing the top-15 literature checks used by LE-MDCAP. Results are reported over five seeds.
The decisive check is the asymmetry predicted by the core claim. Removing the association subspace should raise causal-versus-non-causal AUROC while leaving causal-versus-unrelated roughly flat or slightly lower, mirroring the trade-off LE-MDCAP and DNI-MDCAP observed between these two tasks. If removal hurts both tasks, the text embeddings carry no causal signal separable from association, and the proposal's premise fails. A second check is the strict leakage variant: gains that vanish once co-mentioning abstracts are dropped would mean the model was reading labels, not predicting them. A third check stratifies performance by entity popularity. A genuinely causal scorer should hold up on rarely studied diseases, whereas a popularity-driven one would collapse there.
References
- A Computational Model to Predict the Causal miRNAs for Diseases
- Annotation and curation of the causality information in LncRNADisease
- LE-MDCAP: A Computational Model to Prioritize Causal miRNA-Disease Associations
- DNI-MDCAP: improvement of causal MiRNA-disease association prediction based on deep network imputation
- Benchmark and Best Practices for Biomedical Knowledge Graph Embeddings
- Edge: Enriching Knowledge Graph Embeddings with External Text
- Pre-trained Language Models in Biomedical Domain: A Systematic Survey
- Knowledge Graph Embeddings for Causal Relation Prediction
- CausE: Towards Causal Knowledge Graph Embedding
- Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful? A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Tasks
- Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models