Simple Semantic-based Data Augmentation for Named Entity Recognition in Biomedical Texts

Uyen Phan, Nhung NguyenACL 2022

TL;DR

This paper introduces a novel data augmentation technique called Semantic Neighbour Replacement (SNR) to improve Named Entity Recognition (NER) in biomedical texts. By focusing on preserving the semantic meaning of entities and sentences, SNR effectively boosts NER performance, especially in low-resource scenarios, outperforming existing methods.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is a crucial task in natural language processing, aiming to identify and categorize named entities (like diseases, drugs, or symptoms) within text. However, building high-performing NER models often requires large amounts of labeled data, which is expensive and time-consuming to create, particularly in specialized domains like biomedicine. Data augmentation, a technique to artificially increase the size and diversity of training data, is a promising solution.

The challenge with data augmentation for NER, especially in the biomedical field, is that simple text manipulation methods (like swapping or deleting words) can easily break the semantic integrity of named entities and the overall sentence meaning. For instance, changing a single word within a drug name could render it nonsensical or misidentified. Existing methods like Mention Replacement (MR) can be too random, leading to semantically incorrect augmented sentences. This paper tackles the problem of creating effective, semantically-preserving data augmentations for biomedical NER.

Methods

The proposed method, Semantic Neighbour Replacement (SNR), operates in three phases:

  1. Semantic Neighbour Extraction: This phase leverages pre-trained language models (like ClinicalBERT for English and PhoBERT for Vietnamese) to represent entities as embeddings. It then identifies "semantic neighbors" for an entity: other entities of the same type that are semantically similar, based on cosine similarity of their embeddings and a threshold parameter ($\alpha$).
  2. Entity Replacement: Here, an original entity in a sentence is replaced by one of its identified semantic neighbors. This process is applied to one entity of each type within a sentence, generating a set of augmented sentences.
  3. Sentence Evaluation: To ensure the augmented sentences retain their original meaning, this phase uses a sentence-level language model (Sentence-BERT) to embed both original and augmented sentences. The cosine similarity between them is calculated, and sentences falling below a similarity threshold ($\theta$) are discarded. This step is crucial for filtering out semantically incorrect augmentations.

The authors also compare SNR against two other methods:

  • Mention Replacement (MR): A simpler method that replaces entities randomly.
  • Entity Replacement (ER): This is essentially SNR without the final Sentence Evaluation phase, allowing potentially semantically incorrect sentences to be included.

Datasets

The experiments were conducted on two biomedical datasets:

  • i2b2-2010: An English corpus of clinical records containing entities for Medical Problem, Test, and Treatment.
  • VietBioNER: A Vietnamese corpus of biomedical texts related to tuberculosis, annotated with entities for Organisation, Location, Date and Time, Symptom and Disease, and Diagnostic Procedure.

To simulate low-resource settings, the authors created small (50 sentences), medium (150 sentences), and large (500 sentences) subsets from the training data, in addition to using the full training sets.

Key Insights and Ideas

  • Semantics are King for NER Augmentation: The core insight is that simply augmenting text isn't enough for NER. Preserving the semantic meaning of both individual entities and the entire sentence is paramount for effective augmentation.
  • Targeted Augmentation Outperforms Randomness: SNR's success stems from its ability to intelligently select replacements that are semantically related, unlike purely random methods like MR.
  • Filtering is Crucial: The Sentence Evaluation phase in SNR is vital. Even though it discards a significant portion of generated sentences, the remaining ones are more semantically coherent, leading to better downstream NER performance. This highlights that quality of augmentation matters more than sheer quantity.
  • Biomedical Domain Specificity: The paper demonstrates the effectiveness of SNR on specialized biomedical datasets, suggesting its potential for other domain-specific NLP tasks.

Key Results

  • Improved NER Performance: SNR consistently improved NER performance (measured by F1-scores) across both English (i2b2-2010) and Vietnamese (VietBioNER) corpora, even when using the full training datasets.
  • Significant Gains in Low-Resource Settings: The method showed notable improvements, particularly in the small and medium dataset settings, demonstrating its utility for data-scarce scenarios.
  • SNR Outperforms Baselines and ER: SNR generally outperformed the baseline (no augmentation) and the Entity Replacement (ER) method, which lacked the semantic filtering. For i2b2-2010, SNR achieved an average F1 improvement of 1.23%, and for VietBioNER, it was 0.84%.
  • Efficiency of SNR: Interestingly, SNR generated fewer augmented sentences than ER (around 37% fewer in one case) but achieved better NER performance, reinforcing the "quality over quantity" principle.

Limitations and Future Work

  • In-Domain Augmentation: The current SNR method only augments data within the same domain. The authors suggest future work could explore cross-domain augmentation, leveraging rich-resource data to enrich low-resource domains.
  • Parameter Tuning: The effectiveness of SNR relies on careful tuning of the semantic similarity thresholds ($\alpha$ and $\theta$). While the paper provides specific values used, optimal settings might vary across different datasets and tasks.
  • Limited Entity Types: For VietBioNER, augmentation was only applied to specific entity types (Symptom_and_Disease, Diagnostic Procedure). Future work could extend this to all entity types.

Summary updated

BibTeX
@inproceedings{phan2022simple,
  author = {Uyen Phan and Nhung Nguyen},
  title = {{Simple Semantic-based Data Augmentation for Named Entity Recognition in Biomedical Texts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure