GraphPrompt: Graph-Based Prompt Templates for Biomedical Synonym Prediction

Hanwen Xu, Jiayou Zhang, Zhirui Wang, Shizhuo Zhang, Megh Bhalerao, Yucong Liu, Dawei Zhu, Sheng WangAAAI 2023 · OpenReview

TL;DR

This paper introduces GraphPrompt, a novel method for biomedical synonym prediction that leverages the rich relational structure of ontologies. By converting graph information into prompt templates for language models, GraphPrompt significantly improves accuracy, especially for challenging zero-shot and few-shot scenarios, and introduces a new, large-scale biomedical dataset (OBO-syn) to facilitate research in this area.

What Problem Does This Paper Address?

The biomedical field is awash with data, but a persistent challenge is the inconsistent terminology used to describe the same concepts. This means that a single entity, like a biological process or a cellular component, might be referred to by many different names across scientific literature and databases. Automatically mapping these synonyms to a standardized concept (biomedical synonym prediction or entity normalization) is crucial for tasks like summarizing research, comparing findings, and validating discoveries.

Existing methods often rely on textual context or general language models. However, for certain types of biomedical entities (e.g., functions, pathways), contextual clues are scarce, and these entities are often under-explored in existing datasets. Furthermore, these methods often fail to utilize the inherent structural relationships within biomedical ontologies, where entities are interconnected through various relationships (like "is_a" or "part_of"). This paper aims to bridge this gap by developing a method that effectively incorporates these graph structures into synonym prediction, particularly for challenging cases with limited or no training data.

Methods

The core innovation of GraphPrompt lies in its unique approach to integrating graph structures with prompt-based learning. Instead of directly feeding graph embeddings into a model or using graph neural networks on text, GraphPrompt transforms the graph's relational information into text-based prompt templates.

Here's a breakdown:

  • Base Model: A standard encoder-decoder architecture (using BioBERT) is established to calculate similarity between a queried synonym and candidate entities based solely on their textual representations. This serves as a baseline.
  • Prompt Model: This model refines the base by using prompt templates (e.g., "[MASK] is identical with [entity]") to represent entities and synonyms. This allows a masked language model (MLM) task to learn representations, leveraging the power of pre-trained language models.
  • GraphPrompt: This is the main contribution. It extends the prompt model by incorporating graph information.
  • First-order GraphPrompt: It creates prompt templates based on immediate neighbors in the ontology graph (e.g., "entity_A [relation] entity_B").
  • Second-order GraphPrompt: It further expands this by considering neighbors of neighbors, creating templates that capture more distant but still relevant relationships (e.g., "entity_A [relation] entity_B, which is a kind of entity_C").
  • Contrastive Loss: An additional loss term is introduced to encourage the model to distinguish between different entities while increasing the similarity of related entities, further refining the learned representations.

The key novelty is the conversion of graph topology into natural language prompts, allowing pre-trained language models to "understand" and utilize these relationships without explicit graph embedding layers.

Datasets

The paper introduces a new, large-scale dataset called OBO-syn.

  • OBO-syn: This dataset is curated from the Open Biological and Biomedical Ontology (OBO) Foundry and encompasses 70 different types of biomedical entities. It contains over 2 million curated entity-synonym pairs.
  • Key Differentiator: Unlike existing datasets, OBO-syn explicitly leverages the relational graph structure inherent in ontologies, where nodes (entities) are connected by edges (relations). This structural information is central to GraphPrompt's approach.
  • Challenges: The dataset exhibits a substantial number of out-of-vocabulary phrases and has smaller overlaps with existing datasets, making it a more challenging benchmark for current methods.

The authors also evaluated their method on existing biomedical entity normalization datasets, though the primary focus and novelty lie with OBO-syn.

Key Insights and Ideas

  • Graph Structure as Language: The most significant insight is that the relational structure of ontologies can be effectively translated into natural language prompts. This allows powerful pre-trained language models to leverage this structural information without needing complex graph-specific architectures.
  • Beyond Textual Similarity: For many biomedical entities, pure textual similarity is insufficient. The graph structure provides crucial semantic links between entities that might otherwise appear dissimilar, especially in zero-shot scenarios where no direct training examples exist.
  • Higher-Order Neighbors Matter: Considering second-order neighbors (neighbors of neighbors) in the graph significantly boosts performance. This suggests that indirect relationships within an ontology carry valuable information for synonym prediction.
  • OBO-syn as a Challenging Benchmark: The new OBO-syn dataset, with its emphasis on graph structure and a higher proportion of less common terms, highlights the limitations of existing methods and the need for graph-aware approaches.

Key Results

GraphPrompt demonstrated superior performance across multiple datasets and settings:

  • Few-shot Setting: GraphPrompt achieved an average improvement of 27.7% on Acc@10 and 35.5% on Acc@1 compared to the best-performing synonym prediction approach (BioSyn).
  • Zero-shot Setting: The improvement was even more pronounced in the zero-shot setting, where GraphPrompt consistently outperformed other methods, underscoring the importance of graph information when direct training data is absent.
  • Comparison to GCN: GraphPrompt consistently outperformed a graph convolutional network (GCN) approach, suggesting that its prompt-based method is less susceptible to the "over-smoothing" issue common in GCNs, especially for entities closer to the graph's center.
  • Effectiveness of Lc: The contrastive loss term (Lc) significantly improved performance, particularly in the few-shot setting, highlighting the benefit of explicitly learning to differentiate between entities.

Limitations and Future Work

The authors acknowledge several limitations and propose future directions:

  • Entities Not in Graph: GraphPrompt currently requires entities to be present within a predefined graph. For novel entities not in the ontology, performance might be hindered.
  • Future Work: The authors plan to supplement entity representations with textual descriptions (e.g., from the "def" field in OBO) to help bridge this gap and connect novel entities to existing ones.
  • Simple Relation Descriptions: The current study uses relatively simple text representations for relations.
  • Future Work: Incorporating richer textual descriptions of relations, potentially mined from literature, could further enhance performance and applicability to more complex networks.
  • Broader Applications: The authors are interested in extending GraphPrompt to other graph-based NLP tasks, such as citation network analysis and graph-based text generation. They also envision OBO-syn being useful for tasks beyond entity normalization, including link prediction and named entity recognition.

Summary updated

Abstract

In the expansion of biomedical dataset, the same category may be labeled with different terms, thus being tedious and onerous to curate these terms. Therefore, automatically mapping synonymous terms onto the ontologies is desirable, which we name as biomedical synonym prediction task. Unlike biomedical concept normalization (BCN), no clues from context can be used to enhance synonym prediction, making it essential to extract graph features from ontology. We introduce an expert-curated dataset OBO-syn encompassing 70 different types of concepts and 2 million curated concept-term pairs for evaluating synonym prediction methods. We find BCN methods perform weakly on this task for not making full use of graph information. Therefore, we propose GraphPrompt, a prompt-based learning approach that creates prompt templates according to the graphs. GraphPrompt obtained 37.2% and 28.5% improvement on zero-shot and few-shot settings respectively, indicating the effectiveness of these graph-based prompt templates. We envision that our method GraphPrompt and OBO-syn dataset can be broadly applied to graph-based NLP tasks, and serve as the basis for analyzing diverse and accumulating biomedical data. All the data and codes are avalible at: https://github.com/HanwenXuTHU/GraphPrompt
BibTeX
@inproceedings{xu2023graphprompt,
  author = {Hanwen Xu and Jiayou Zhang and Zhirui Wang and Shizhuo Zhang and Megh Bhalerao and Yucong Liu and Dawei Zhu and Sheng Wang},
  title = {{GraphPrompt: Graph-Based Prompt Templates for Biomedical Synonym Prediction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=JB5C2XgeCp},
}
Figure