Abstract
Understanding the meaning of text often involves reasoning about entities and their relationships. This requires identifying textual mentions of entities, linking them to a canonical concept, and discerning their relationships. These tasks are nearly always viewed as separate components within a pipeline, each requiring a distinct model and training data. While relation extraction can often be trained with readily available weak or distant supervision, entity linkers typically require expensive mention-level supervision – which is not available in many domains. Instead, we propose a model which is trained to simultaneously produce entity linking and relation decisions while requiring no mention-level annotations. This approach avoids cascading errors that arise from pipelined methods and more accurately predicts entity relationships from text. We show that our model outperforms a state-of-the art entity linking and relation extraction pipeline on two biomedical datasets and can drastically improve the overall recall of the system.
Introduction
In the biomedical field, converting raw text, such as PubMed abstracts, into structured knowledge graphs is a critical but challenging task. Traditionally, this process follows a pipeline architecture: first, Named Entity Recognition (NER) identifies mentions of chemicals or diseases; second, Entity Linking (EL) maps these mentions to unique identifiers in a database; and finally, Relation Extraction (RE) determines the relationships between these linked entities.
However, this pipelined approach suffers from two major drawbacks. First, it is prone to cascading errors: an error in the entity linking stage inevitably leads to failure in relation extraction. Second, training effective entity linkers usually requires "mention-level" supervision—expensive, expert-annotated labels for every word in a corpus. In specialized domains like biomedicine, such data is often unavailable.
Methodology: The SNERL Model
To address these issues, researchers from UMass Amherst and Google Research developed SNERL (Simultaneous Neural Entity-Relation Linker). SNERL treats entity linking and relation extraction as a joint probabilistic task, eliminating the need for a rigid pipeline.
The model's architecture consists of several key components:
- Text Encoding: A Transformer-based encoder, augmented with convolutions to capture local context, generates representations for every word in the input text.
- Joint Prediction: For every potential entity mention, the model generates a set of candidates. It then calculates the probability of a "triple" (Entity 1, Relation, Entity 2) by multiplying the probabilities of the individual entity links and the relationship between them.
- Distant Supervision: SNERL is trained using document-level distant supervision. It does not require knowing exactly where a relation is mentioned in the text; it only needs to know that a specific document is associated with a fact in a database.
- Soft Pooling: To aggregate evidence across a document, the model employs "soft maximum pooling," allowing it to learn from multiple mentions of the same entity while remaining robust to noise.
Datasets and Evaluation
The model was evaluated on three diverse datasets:
- Chemical Toxicology Database (CTD): A large-scale dataset focusing on relationships between chemicals, diseases, and genes.
- Chemical Disease Relation (CDR): A gold-standard benchmark used to validate the entity linking component.
- Disease-Phenotype Dataset: A new benchmark created from the Human Phenotype Ontology, featuring rare diseases and symptoms where expert-labeled data is scarce.
Results
SNERL demonstrated significant improvements over traditional pipelined methods, particularly in terms of recall. On the full CTD dataset, SNERL achieved an F1 score of 42.2, substantially outperforming the state-of-the-art BRAN pipeline.
The most dramatic gain was observed in recall. On the CTD set, SNERL’s recall reached 43.4%, compared to 28.1% for the pipeline. This suggests that joint modeling prevents the "leakage" of information that occurs when pipelines discard potential candidates too early. Furthermore, even without direct mention-level supervision, SNERL achieved an 86.6 F1 score on the CDR entity linking task, surpassing standard string-matching baselines.
Discussion and Key Insights
A major takeaway from this research is that Relation Extraction helps Entity Linking. By modeling these tasks jointly, the context of a potential relationship helps disambiguate entities. For instance, if a sentence describes a "causes" relationship, the model will favor entity candidates known to participate in such relations.
By moving away from mention-level labels, the researchers demonstrated that high-performing models can be built using only the "weak" labels already present in existing medical databases. This effectively eliminates a major bottleneck in biomedical knowledge extraction.
Limitations and Future Work
Despite its success, SNERL faces some limitations:
- Candidate Generation: The model's performance is capped by its initial string-similarity-based candidate generation. If the correct entity is not among the top candidates, the model cannot recover it.
- Out-of-Vocabulary Entities: Handling rare or unseen entities remains a challenge.
- Computational Complexity: Evaluating all possible relations between all candidates in a document is expensive. While top-k pruning was used, further optimizations are needed for scaling to very large documents.
Future research may involve incorporating sophisticated entity embeddings from knowledge graphs to improve reasoning about rare or unseen entities.
Summary updated