Abstract
The exponential growth of biomedical literature presents a significant challenge for researchers attempting to synthesize knowledge manually. This paper introduces an open-source, end-to-end framework designed to transform unstructured biomedical text into structured knowledge graphs. By leveraging automated data pipelines and transformer-based models, the framework identifies complex semantic relations between medical entities. Applied to Rett Syndrome and Alzheimer’s Disease, the system produced high-quality annotated datasets and achieved near-human performance in relation detection, demonstrating its potential for accelerating discovery in both common and rare diseases.
Introduction
The volume of biomedical research is increasing at an unprecedented rate, making manual curation of knowledge bases unsustainable. While some diseases benefit from structured databases, many—particularly rare conditions like Rett Syndrome—lack comprehensive, machine-readable resources.
Existing Information Extraction (IE) tools often target specific interactions, such as protein-protein or drug-drug pairs. There is a critical need for disease-agnostic frameworks capable of identifying a broad spectrum of entities (e.g., symptoms, genes, chemicals) and the nuanced semantic relationships between them. This work addresses this gap by providing a pipeline to convert unstructured text into actionable knowledge.
Methodology
The proposed framework consists of a multi-stage data pipeline and specialized model architectures for relation detection.
Data Pipeline
The pipeline automates the transition from raw literature to structured datasets through four primary stages:
- Abstract Retrieval: Automated fetching of PubMed abstracts using the Entrez Programming Utilities.
- Mention Extraction: Utilizing MetaMapLite to identify and link text mentions to the Unified Medical Language System (UMLS), covering 82 semantic types.
- Co-occurrence Graph Generation: Identifying potential entity pairs based on intra-sentence co-occurrence to focus annotation efforts.
- Dataset Creation: Generating "gold-standard" datasets through expert annotation and "silver-standard" datasets via distant supervision.
Model Architectures
Two primary architectures were evaluated: LaMEL (Language Model Embedding Learning), which uses cosine similarity in an optimized embedding space, and LaMReD (Language-Model Relation Detection), a classification-based approach using feature aggregation. Probing experiments using BiomedBERT revealed that the 10th and 11th layers of the transformer are most critical for capturing semantic relations.
Datasets and Results
The framework was validated using two new gold-standard datasets: ReDReS (Rett Syndrome) and ReDAD (Alzheimer’s Disease). Additionally, massive silver-standard datasets (DiSReDReS and DiSReDAD) were created using distant supervision, providing millions of instances for training future weakly supervised models.
Performance Metrics
The models demonstrated high accuracy across different tasks:
- Binary Classification: Achieved F1-scores of approximately 91%, nearing the human expert performance of 92%.
- Multi-class Classification: For distinguishing between Positive, Negative, Complex, and No Relation, models reached a Macro F1-score of ~78%.
- Cross-Disease Robustness: Models trained on Alzheimer’s data performed effectively on Rett Syndrome data, indicating strong domain transferability.
Discussion and Conclusion
A key finding was that encoder-based models like BiomedBERT outperformed larger decoder-based models like BioGPT for these discriminative tasks. The inclusion of the [CLS] token and the text context between entities proved vital for performance.
Despite these successes, challenges remain in handling class imbalance—specifically the rarity of "Negative" relations—and distinguishing between "Positive" and "Complex" relationships that require deeper reasoning. By open-sourcing the framework, the authors aim to empower the research community to build knowledge bases for a wide array of under-researched diseases.
Summary updated