A Multi-Task Approach for Improving Biomedical Named Entity Recognition by Incorporating Multi-Granularity information

Yiqi Tong, Yidong Chen, Xiaodong ShiACL 2021

TL;DR

This paper tackles the challenge of limited annotated data in biomedical named entity recognition (BioNER) by proposing a multi-task learning approach. By jointly training the main NER task with auxiliary tasks that leverage information at different granularities (sentence-level and multi-token), the model significantly improves performance, especially in low-resource settings.

What Problem Does This Paper Address?

Biomedical Named Entity Recognition (BioNER) is crucial for extracting valuable information like genes, diseases, and chemicals from unstructured text, which fuels downstream NLP tasks. However, creating large, high-quality BioNER datasets is a significant hurdle due to privacy concerns, ethical considerations, and the highly specialized nature of the domain. This scarcity of annotated data makes it difficult to train robust BioNER models. While existing methods often rely solely on token-level information, this paper aims to alleviate the data bottleneck by exploiting implicit information present at different levels of granularity within existing datasets.

Methods

The core of this work is a multi-task learning framework built upon the BioBERT pre-trained language model. Instead of just training for the primary NER task (identifying entities at the token level), the authors introduce three auxiliary tasks:

  • Binary Classification (bCLS): Predicts whether a sentence contains any entities. This acts as a global guidance signal.
  • Multi-Class Classification (mCLS): Predicts the number of entities in a sentence (categorized into 0, 1, 2, or 3+ entities). This helps alleviate under- or over-recognition issues.
  • Multi-Token Classification (mtCLS): Predicts whether a token belongs to a multi-token entity, a single-token entity, or neither. This specifically targets the challenge of accurately identifying entity boundaries.

These auxiliary tasks are designed to leverage coarse-grained information (sentence-level) and finer-grained information (multi-token) that are implicitly present in the NER labels, thus requiring no additional manual annotations. The BioBERT encoder is shared across all tasks, with task-specific layers for each objective. The final loss is a weighted sum of the losses from all tasks.

Datasets

The experiments were conducted on three established biomedical NER datasets:

  • BC2GM: Focused on gene/protein entities.
  • BC5CDR-chem: Focused on chemical entities.
  • NCBI-disease: Focused on disease entities.

The paper does not introduce new datasets but rather leverages existing ones to demonstrate the effectiveness of their multi-task approach.

Key Insights and Ideas

  • Multi-Granularity Information is Key: The paper strongly suggests that leveraging information beyond just token-level labels (i.e., sentence-level and multi-token context) can significantly boost BioNER performance, especially when data is scarce.
  • Auxiliary Tasks Complement Each Other: Different auxiliary tasks contribute to the main NER task in varying ways. For instance, mtCLS is particularly beneficial for BC2GM (likely due to entity boundary challenges), while bCLS is more impactful for BC5CDR-chem and NCBI-disease (suggesting entity sparsity as a key issue in those datasets).
  • Low-Resource Performance Boost: The proposed method shows a marked improvement in low-resource scenarios, indicating its potential for practical applications where extensive annotation is infeasible.
  • Task Relationships Matter: The study hints at complex interactions between tasks, with some combinations yielding better results than others, and performance varying across datasets. This suggests that careful selection and tuning of auxiliary tasks can be crucial.

Key Results

  • The proposed multi-task model, particularly when using max-pooling for sentence-level tasks (Ours_MAX), achieved state-of-the-art or near state-of-the-art performance on the BC2GM, BC5CDR-chem, and NCBI-disease datasets.
  • Improvements of up to 0.91 F1-score were observed over the BioBERT baseline.
  • In low-resource settings (e.g., 10% of the training data for NCBI-disease), the model achieved up to a 3.29 F1-score improvement over BioBERT, demonstrating its effectiveness in data-scarce environments.
  • Ablation studies confirmed that removing any of the auxiliary tasks or the joint training strategy led to a performance drop, underscoring the value of the proposed approach.

Limitations and Future Work

  • Task Relationship Complexity: While the paper explores task relationships, a deeper understanding and more efficient methods for characterizing these relationships are needed. The authors suggest future work could explore efficient methods for generating vectorial representations to measure task relationships.
  • Computational Resources: Multi-task learning, especially with extensive experimentation, can be computationally intensive.
  • Generalizability of Task Combinations: The effectiveness of specific task combinations can vary across datasets, suggesting that domain-specific considerations might be important for optimal performance.
  • Future Directions: The authors plan to explore more efficient methods for understanding task relationships and potentially incorporate domain-specific knowledge more deeply into future models.

Summary updated

BibTeX
@inproceedings{tong2021multi,
  author = {Yiqi Tong and Yidong Chen and Xiaodong Shi},
  title = {{A Multi-Task Approach for Improving Biomedical Named Entity Recognition by Incorporating Multi-Granularity information}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure