TL;DR
This paper adapts a novel framework for extracting scientific knowledge from text using distant supervision, a method that leverages existing knowledge graphs to automatically label data. They demonstrate its feasibility on biomedical data and introduce extensions, including incorporating entity type information, to significantly improve relation extraction performance.
What Problem Does This Paper Address?
The explosion of scientific literature makes it challenging to extract and structure valuable knowledge. Scientific Knowledge Graphs (KGs), like UMLS, are crucial for many NLP tasks, but they are often incomplete. Relation Extraction (RE) aims to fill these gaps by identifying relationships between entities in text. A common approach is "distant supervision," which automatically labels sentences containing entity pairs found in a KG. However, this method suffers from "wrong labeling" – sentences might not actually express the intended relation, leading to noisy training data and reduced RE performance. This paper tackles the challenge of applying and improving distant supervision specifically for the complex domain of biomedical literature.
Methods
The core of this work is an adaptation and extension of a previously proposed "joint model" that trains a Relation Extraction (RE) model and a Knowledge Graph Completion (KGC) model together.
- Base Model: The authors use a framework (Han et al., 2018) that combines a KG-based attention mechanism with a RE model. This attention mechanism helps the RE model focus on more informative sentences within a "bag" of sentences that are distantly labeled for a given entity pair. The KGC component learns embeddings for entities and relations in the KG.
- KGC Extensions: The base model initially used KGC models based on translational distance (TransE, TransD). The authors extend this by incorporating "semantic matching" KGC models (ComplEx, SimplE), hypothesizing they might better capture the nuances of biomedical relations.
- End-to-End KGC with Entity Type (ET) Information: A key novelty is a proposed end-to-end KGC model (dubbed SimplE_NER) that integrates entity type (ET) information. In biomedical contexts, ETs are crucial (e.g., a drug treats a disease). This model uses word embeddings to predict ETs and then incorporates them into the KGC scoring function, aiming to improve both KGC and RE performance.
Datasets
- Knowledge Graph: UMLS (Unified Medical Language System) – a large biomedical KG. They extracted RO (Relationship Other than synonymous, narrower, or broader) relation triplets.
- Textual Data: Medline corpus – a collection of biomedical abstracts. They extracted sentences containing entity pairs from the KG for training and testing the RE model. They also collected entity-ET pairs from UMLS for training the ET classification part of their KGC model.
Key Insights and Ideas
- Feasibility of Joint Models in Biomedicine: The paper confirms that the distant supervision framework, which jointly trains RE and KGC, is not only feasible but also effective for the biomedical domain, outperforming models trained solely on text.
- KGC Model Choice Matters: The performance of the joint RE-KGC model is sensitive to the choice of KGC model. Semantic matching models (ComplEx, SimplE) showed promise over translational models for this specific biomedical task.
- Entity Type is Crucial: Incorporating entity type information into the KGC model significantly boosts performance. This highlights the importance of domain-specific features for biomedical knowledge extraction.
- KGC Link Prediction as a Proxy: The authors suggest that KGC model performance on link prediction tasks can be a good indicator of its effectiveness for distantly supervised RE.
Key Results
The extended models achieved significant and consistent improvements in distantly supervised RE performance compared to baselines. Specifically:
- Models with KG-based attention ("JointE+KATT") outperformed those without it ("CNN+ATT").
- The extended models ("JointComplEx+KATT", "JointSimplE+KATT", "JointSimplE_NER+KATT") showed superior precision across a wide range of recall compared to the base models.
- The proposed "JointSimplE_NER+KATT" model achieved the highest precision overall, demonstrating the effectiveness of incorporating entity type information.
Limitations and Future Work
- Parameter Tuning: The authors mention that for this study, they used default parameters for the base model and will tune them further in future work.
- Exploring Other Semantic Features: The success of ET information suggests that other semantic features, such as entity definitions or "fact triplet chains" (sequences of relations), could be explored to further enhance the model.
- Dataset Specificity: While the base model's performance varied between scientific and non-scientific datasets, the extensions showed consistent improvements on the biomedical data. Future work could explore how different KGC models perform on various scientific domains.
Summary updated