Improving Biomedical Analogical Retrieval with Embedding of Structural Dependencies

Amandalynne Paullada, Bethany Percha, Trevor CohenACL 2020

TL;DR

This paper introduces a novel method, Embedding of Structural Dependencies (ESD), that incorporates syntactic dependency information into word embeddings to improve the retrieval of biomedical relationships. By capturing the structural relationships fundamental to analogy, ESD significantly outperforms standard word embeddings (SGNS) on both retrieving known relationships and discovering novel ones from biomedical literature.

What Problem Does This Paper Address?

The biomedical field is characterized by a rapid influx of new research, making it challenging to keep human-curated knowledge bases up-to-date. This creates a need for automated methods to infer and discover relationships between biomedical entities (like drugs, genes, and diseases) directly from scientific literature. While existing neural word embedding techniques can capture some relational information, they often overlook the crucial role of syntax in understanding complex relationships, which is a cornerstone of human analogical reasoning. This limitation hinders their ability to accurately represent and retrieve nuanced biomedical connections.

Methods

The core contribution of this paper is the Embedding of Structural Dependencies (ESD) method. ESD builds upon existing word embedding techniques but crucially incorporates syntactic dependency information. Specifically, it represents relationships as dependency paths extracted from parsed sentences. These paths, which describe the grammatical relationships between entities, are then encoded into embeddings.

The authors compare ESD to Skip-gram with Negative Sampling (SGNS), a standard word embedding model. While SGNS learns embeddings from co-occurrence statistics in raw sentences, ESD explicitly encodes the structural relationships found in dependency parses. The paper adapts an existing method, Embedding of Semantic Predications (ESP), to create ESD, using dependency path strings as "predicate tokens" and employing a binary vector representation with a non-negative normalized Hamming distance for similarity.

Datasets

The study utilizes a large corpus of approximately 70 million sentences from biomedical research paper abstracts from Medline for training the embedding models.

For evaluation, the authors construct datasets from 19 existing biomedical knowledge bases. These knowledge bases cover four primary types of relationships: chemical-gene, chemical-disease, gene-disease, and gene-gene. The evaluation is split into two tasks:

  • Relationship Retrieval (RR): This task uses knowledge base pairs that are connected by at least one dependency path in the training corpus. The goal is to retrieve known relationships.
  • Literature-Based Discovery (LBD): This task uses knowledge base pairs that are not connected by any dependency path in the training corpus. This serves as a proxy for discovering novel, implicit relationships.

Key Insights and Ideas

  • Syntax Matters for Biomedical Analogy: The paper strongly suggests that simply relying on word co-occurrence (as in SGNS) is insufficient for capturing complex biomedical relationships. Incorporating syntactic structure, specifically dependency paths, significantly enhances the ability to represent and retrieve these relationships.
  • Structural Embeddings Excel at Analogy: ESD's ability to encode dependency paths allows it to better solve proportional analogy problems, a key indicator of relational understanding. This is crucial for tasks like inferring treatments or identifying side effects.
  • Beyond Distributional Similarity: The comparison between "full analogy" (A:B::C:D) and "B:D" (direct similarity) baselines reveals that ESD often performs better with the full analogy, indicating it's not just relying on simple term similarity but on learned relational patterns.
  • LBD is Harder, but ESD Still Leads: Literature-based discovery, which involves finding novel relationships, is inherently more challenging. However, ESD consistently outperforms SGNS even on this difficult task, highlighting its potential for hypothesis generation.

Key Results

ESD significantly outperforms SGNS across the board:

  • Relationship Retrieval (RR): ESD outperforms SGNS on 18 out of 19 knowledge bases.
  • Literature-Based Discovery (LBD): ESD outperforms SGNS on 17 out of 19 knowledge bases.
  • Qualitative Analysis: In a specific example involving risperidone (an antipsychotic drug), ESD retrieved more correct or plausible treatment targets, whereas SGNS primarily returned other drugs or side effects. Crucially, ESD's retrieved results were all of the correct semantic type (disorders), while SGNS returned a mix of drugs and side effects.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the LBD task is made more difficult by the lack of consistent normalization of concepts across different knowledge bases.
  • For the SIDER side effects dataset, the B:D baseline for ESD showed higher scores than the full analogy, suggesting that for some specific relationship types (like side effects with high overlap), distributional similarity might play a larger role.
  • Future Work:
    • The authors propose comparing their methods with knowledge base completion techniques that utilize contextualized vectors from language models (e.g., ELMo).
  • They also plan to explore the application of these vector space models to analogies in the general domain.

Summary updated

BibTeX
@inproceedings{paullada2020improving,
  author = {Amandalynne Paullada and Bethany Percha and Trevor Cohen},
  title = {{Improving Biomedical Analogical Retrieval with Embedding of Structural Dependencies}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
}
Figure