Distantly Supervised Biomedical Knowledge Acquisition via Knowledge Graph Based Attention

Qin Dai, Naoya Inoue, Paul Reisert, Ryo Takahashi, Kentaro Inui2019 · DOI

TL;DR

This paper adapts a novel framework for extracting scientific knowledge from text using distant supervision, a method that leverages existing knowledge graphs to automatically label data. They demonstrate its feasibility on biomedical data and introduce extensions, including incorporating entity type information, to significantly improve relation extraction performance.

What Problem Does This Paper Address?

The explosion of scientific literature makes it challenging to extract and structure valuable knowledge. Scientific Knowledge Graphs (KGs), like UMLS, are crucial for many NLP tasks, but they are often incomplete. Relation Extraction (RE) aims to fill these gaps by identifying relationships between entities in text. A common approach is "distant supervision," which automatically labels sentences containing entity pairs found in a KG. However, this method suffers from "wrong labeling" – sentences might not actually express the intended relation, leading to noisy training data and reduced RE performance. This paper tackles the challenge of applying and improving distant supervision specifically for the complex domain of biomedical literature.

Methods

The core of this work is an adaptation and extension of a previously proposed "joint model" that trains a Relation Extraction (RE) model and a Knowledge Graph Completion (KGC) model together.

  • Base Model: The authors use a framework (Han et al., 2018) that combines a KG-based attention mechanism with a RE model. This attention mechanism helps the RE model focus on more informative sentences within a "bag" of sentences that are distantly labeled for a given entity pair. The KGC component learns embeddings for entities and relations in the KG.
  • KGC Extensions: The base model initially used KGC models based on translational distance (TransE, TransD). The authors extend this by incorporating "semantic matching" KGC models (ComplEx, SimplE), hypothesizing they might better capture the nuances of biomedical relations.
  • End-to-End KGC with Entity Type (ET) Information: A key novelty is a proposed end-to-end KGC model (dubbed SimplE_NER) that integrates entity type (ET) information. In biomedical contexts, ETs are crucial (e.g., a drug treats a disease). This model uses word embeddings to predict ETs and then incorporates them into the KGC scoring function, aiming to improve both KGC and RE performance.

Datasets

  • Knowledge Graph: UMLS (Unified Medical Language System) – a large biomedical KG. They extracted RO (Relationship Other than synonymous, narrower, or broader) relation triplets.
  • Textual Data: Medline corpus – a collection of biomedical abstracts. They extracted sentences containing entity pairs from the KG for training and testing the RE model. They also collected entity-ET pairs from UMLS for training the ET classification part of their KGC model.

Key Insights and Ideas

  • Feasibility of Joint Models in Biomedicine: The paper confirms that the distant supervision framework, which jointly trains RE and KGC, is not only feasible but also effective for the biomedical domain, outperforming models trained solely on text.
  • KGC Model Choice Matters: The performance of the joint RE-KGC model is sensitive to the choice of KGC model. Semantic matching models (ComplEx, SimplE) showed promise over translational models for this specific biomedical task.
  • Entity Type is Crucial: Incorporating entity type information into the KGC model significantly boosts performance. This highlights the importance of domain-specific features for biomedical knowledge extraction.
  • KGC Link Prediction as a Proxy: The authors suggest that KGC model performance on link prediction tasks can be a good indicator of its effectiveness for distantly supervised RE.

Key Results

The extended models achieved significant and consistent improvements in distantly supervised RE performance compared to baselines. Specifically:

  • Models with KG-based attention ("JointE+KATT") outperformed those without it ("CNN+ATT").
  • The extended models ("JointComplEx+KATT", "JointSimplE+KATT", "JointSimplE_NER+KATT") showed superior precision across a wide range of recall compared to the base models.
  • The proposed "JointSimplE_NER+KATT" model achieved the highest precision overall, demonstrating the effectiveness of incorporating entity type information.

Limitations and Future Work

  • Parameter Tuning: The authors mention that for this study, they used default parameters for the base model and will tune them further in future work.
  • Exploring Other Semantic Features: The success of ET information suggests that other semantic features, such as entity definitions or "fact triplet chains" (sequences of relations), could be explored to further enhance the model.
  • Dataset Specificity: While the base model's performance varied between scientific and non-scientific datasets, the extensions showed consistent improvements on the biomedical data. Future work could explore how different KGC models perform on various scientific domains.

Summary updated

Abstract

The increased demand for structured scientific knowledge has attracted considerable attention in extracting scientific relation from the ever growing scientific publications. Distant supervision is widely applied approach to automatically generate large amounts of labelled data with low manual annotation cost. However, distant supervision inevitably accompanies the wrong labelling problem, which will negatively affect the performance of Relation Extraction (RE). To address this issue, (Han et al., 2018) proposes a novel framework for jointly training both RE model and Knowledge Graph Completion (KGC) model to extract structured knowledge from non-scientific dataset. In this work, we firstly investigate the feasibility of this framework on scientific dataset, specifically on biomedical dataset. Secondly, to achieve better performance on the biomedical dataset, we extend the framework with other competitive KGC models. Moreover, we proposed a new end-to-end KGC model to extend the framework. Experimental results not only show the feasibility of the framework on the biomedical dataset, but also indicate the effectiveness of our extensions, because our extended model achieves significant and consistent improvements on distant supervised RE as compared with baselines.
BibTeX
@misc{dai2019distantly,
  author = {Qin Dai and Naoya Inoue and Paul Reisert and Ryo Takahashi and Kentaro Inui},
  title = {{Distantly Supervised Biomedical Knowledge Acquisition via Knowledge Graph Based Attention}},
  year = {2019},
  doi = {10.18653/v1/w19-2601},
}
Figure