GrantRel: Grant Information Extraction via Joint Entity and Relation Extraction

Junyi Bian, Li Huang, Xiaodi Huang, Hong Zhou, Shanfeng ZhuACL 2021 · OpenReview

TL;DR

This paper introduces GrantRel, a novel framework for automatically extracting grant information (funder names and grant numbers) from academic papers. By combining a funding sentence classifier with a joint entity and relation extraction model that adaptively uses position and context embeddings, GrantRel significantly outperforms existing methods on this challenging task.

What Problem Does This Paper Address?

Identifying grant information within scientific publications is crucial for various stakeholders. Funding bodies need to track their investments, academic institutions want to measure the impact of agencies supporting specific research areas, and literature management systems require this data for cataloging. However, extracting this information presents two main challenges: the lack of high-quality benchmark datasets and the difficulty in capturing complex, often many-to-many, relationships between funders and grant identifiers. Existing methods often struggle with the diversity of funder names and the nuanced ways grant information is presented, sometimes even independently of a specific funder.

Methods

GrantRel employs a two-stage pipeline. First, a funding sentence classifier identifies sentences likely to contain grant information, improving precision and reducing computational cost by filtering out irrelevant sentences. This classifier uses a pre-trained BioBERT model to encode sentence context and a logistic regression layer to predict the probability of a sentence being grant-related.

The core of GrantRel is its joint entity and relation extraction (RE) model. This model leverages BioBERT for encoding input sentences and is designed to simultaneously extract funder names and grant IDs, along with their relationships. It incorporates a funder name tagger that identifies potential funder entities using the IOB tagging scheme. A grant relation detector then links grant IDs to identified funders. Crucially, GrantRel introduces an adaptive embedding mechanism that dynamically balances the influence of positional embeddings (capturing the relative distance between entities) and contextual embeddings (leveraging surrounding text). This adaptive approach allows the model to prioritize context when it's informative and position when context is ambiguous or insufficient, leading to more robust extraction.

Datasets

The authors introduce two new, manually labeled datasets:

  • Grant-RE: This dataset is specifically for training and evaluating the relation extraction module. It contains approximately 3331 sentences and covers nearly 2,000 different funder names, addressing the limitation of existing datasets with a smaller, fixed set of known funders.
  • Grant-SP: This dataset is for training and evaluating the funding sentence classifier. It comprises 1402 sentences, sampled from various sections of papers, and is designed to help the model distinguish grant-related sentences from others.

Both datasets were meticulously annotated by multiple well-trained annotators to ensure high quality. The Grant-RE dataset includes test splits from both PubMed Central (PMC) and arXiv to evaluate the model's generalizability across different scientific domains.

Key Insights and Ideas

  • The power of joint extraction: Treating entity recognition and relation extraction as a unified task, rather than a pipeline, significantly improves performance by allowing the model to leverage shared information.
  • Adaptive embedding is key: The ability to dynamically weigh positional and contextual information is crucial for handling the diverse ways grant information is presented. Context is vital for understanding complex relationships, while position helps when context is less clear or when entities are presented in unusual orders.
  • BioBERT's advantage: Pre-training on scientific literature (BioBERT) provides a significant advantage over general-domain pre-trained models (BERT) for tasks involving scientific text, like grant information extraction.
  • Dataset quality matters: The creation of high-quality, comprehensive datasets like Grant-RE and Grant-SP is essential for advancing research in this area, particularly for capturing the vast number of funder names and complex relationships.

Key Results

GrantRel demonstrates state-of-the-art performance on the Grant-RE dataset.

  • On the PMC test set, GrantRel achieved an F1 score improvement of up to 6.5% over BERT-based RE baselines.
  • On the arXiv test set, the improvement was 3.5%.
  • The adaptive embedding mechanism in GrantRel outperformed simpler fusion methods (e.g., just adding embeddings) by a slight margin, achieving an F1 score of 91.18% compared to 91.03% for the simple fusion approach on the PubMed test set.
  • The sentence classifier component of GrantRel significantly boosted precision, achieving 92.2% F1 score in the pipeline, compared to 76.5% for a keyword-based approach and 21.3% for a pure RE model without sentence filtering.

Limitations and Future Work

  • Error Analysis: The authors identified several error categories, including misclassifying funder entities, incorrectly linking grant IDs to funders, and failing to extract grant IDs when they appear independently. They suggest that more fine-grained tokenization might help correct some of these errors.
  • Data Sparsity: The case study highlighted that the model's performance can be affected by the lack of training examples for certain complex semantic situations or unusual entity presentations.
  • Future Directions: The paper implicitly suggests that further improvements could come from exploring more sophisticated tokenization strategies and potentially incorporating external knowledge bases to better understand the semantic relationships between entities. The generalizability to even more diverse scientific domains could also be a future research avenue.

Summary updated

BibTeX
@inproceedings{bian2021grantrel,
  author = {Junyi Bian and Li Huang and Xiaodi Huang and Hong Zhou and Shanfeng Zhu},
  title = {{GrantRel: Grant Information Extraction via Joint Entity and Relation Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=_8dpuyfzf6b},
}
Figure