Enhancing Biomedical Knowledge Discovery for Diseases: An Open-Source Framework Applied on Rett Syndrome and Alzheimer's Disease

Christos Theodoropoulos, Andrei Catalin Coman, James Henderson, Marie-Francine Moens

TL;DR

This paper introduces an open-source framework for automatically extracting and structuring biomedical knowledge from unstructured text, specifically focusing on diseases like Rett Syndrome and Alzheimer's. By creating novel annotated datasets and employing advanced language models, the research aims to accelerate disease understanding and the development of effective treatments.

What Problem Does This Paper Address?

The sheer volume of biomedical literature is growing exponentially, making it incredibly challenging for researchers to keep up with new findings. This information overload hinders efficient knowledge discovery, particularly for diseases that lack effective cures. The authors tackle this by developing a system that can automatically sift through vast amounts of text to identify and connect key biomedical entities, thereby building a structured knowledge base. This is crucial for accelerating research into complex diseases and ultimately finding cures.

Methods

The core of this work is an end-to-end framework that automates knowledge extraction. The process begins with abstract retrieval from PubMed, followed by mention extraction to identify biomedical entities and link them to the Unified Medical Language System (UMLS). A co-occurrence graph is then generated to map relationships between entities within sentences.

For modeling, the authors introduce two main approaches:

  • LaMEL (Language-Model Embedding Learning): This model focuses on learning an embedding space optimized for relation detection. It uses BiomedBERT as a backbone and explores various ways to represent entities (e.g., using special tokens, pooled embeddings).
  • LaMReD (Language-Model Relation Detection): This approach synthesizes information from the input sequence to construct relation representations. It also uses BiomedBERT and explores different aggregation strategies (element-wise addition or multiplication) for combining token representations.

The framework also incorporates probing experiments to understand how transformer models capture semantic relations by analyzing internal layer representations and attention patterns.

Datasets

The paper introduces two novel annotated datasets:

  • ReDReS (Relation Detection dataset for Rett Syndrome): Contains 601 sentences with 5,259 instances and 1,148 unique CUIs.
  • ReDAD (Relation Detection dataset for Alzheimer's Disease): Contains 641 sentences with 8,565 instances and 1,480 unique CUIs.

These datasets are crucial for benchmarking and evaluating relation detection techniques. Additionally, the authors created distantly supervised versions (DiSReDReS and DiSREDAD) to facilitate weakly supervised learning scenarios.

Key Insights and Ideas

  • The power of structured knowledge: Automatically extracting and structuring information from dense biomedical literature is key to overcoming the information overload and accelerating scientific discovery.
  • Language models as knowledge extractors: Advanced language models like BiomedBERT can effectively capture complex semantic relations within biomedical text, demonstrating their potential for knowledge discovery tasks.
  • Importance of entity and relation representation: The way entities and their relationships are represented significantly impacts the performance of relation detection models. Different strategies yield varying results, highlighting the need for careful selection.
  • Transformer layers hold key information: Probing experiments revealed that specific layers within transformer models (e.g., 10th and 11th) are particularly informative for capturing semantic relations.
  • Cross-disease applicability: The framework's robustness suggests potential for broader applications beyond the specific diseases studied, hinting at transfer learning capabilities.

Key Results

  • The proposed LaMReDA and LaMReDM models, utilizing BiomedBERT, achieved high F1-scores on both ReDReS and ReDAD datasets. For instance, in the binary setup, F1-scores reached up to 91.25% for LaMEL on ReDReS and 91.03% for LaMEL on ReDAD in the 5-fold cross-validation setup.
  • The models demonstrated performance comparable to human experts in binary relation detection tasks.
  • The study found that different relation representations (e.g., RG, RL, RJ, Ro) and entity representations (e.g., EA, ED, EE, EH) had varying impacts on performance, with some (like RG for binary and RL/RJ/Ro for multi-class) showing superior results.
  • The use of the [CLS] token representation was found to be beneficial in the binary setup.
  • The performance of BiomedBERT large generally surpassed the base version, though the base model performed better on original splits in some experiments.
  • The framework showed robustness in cross-disease experiments, indicating its potential for generalization.

Limitations and Future Work

  • Class Imbalance: The datasets exhibit class imbalance, particularly with the under-representation of the "negative relation" class. The authors suggest strategies like undersampling, oversampling, or adjusting loss functions to address this.
  • Complex Relations: Identifying complex semantic relations remains a challenge, with models often misclassifying them as positive or no relation.
  • Future LLMs: The authors suggest evaluating additional and future large language models (LLMs) for the proposed task, further exploring their capabilities in biomedical knowledge discovery.
  • Broader Disease Scope: While the framework is designed to be disease-agnostic, future work could involve applying it to a wider range of diseases to further validate its generalizability.

Summary updated

BibTeX
@article{theodoropoulos2024enhancing,
  author = {Christos Theodoropoulos and Andrei Catalin Coman and James Henderson and Marie-Francine Moens},
  title = {{Enhancing Biomedical Knowledge Discovery for Diseases: An Open-Source Framework Applied on Rett Syndrome and Alzheimer's Disease}},
  journal = {IEEE Access},
  year = {2024},
  doi = {10.1109/access.2024.3509714},
}
Figure