NanoNER: Named Entity Recognition for Nanobiology Using Experts’ Knowledge and Distant Supervision

Ran Cheng, Martin Lentschat, Cyril Labbé

TL;DR

NanoNER is a specialized Named Entity Recognition (NER) model designed to navigate the complex, multidisciplinary landscape of nanobiology. By combining expert-curated ontologies with "distant supervision," the researchers created a high-performing model that achieves a 0.98 F1-score on known terms and successfully discovers new, unseen entities with up to 81% precision—all while requiring minimal manual labeling.

What Problem Does This Paper Address?

In the "data desert" of highly specialized scientific fields, training machine learning models is a catch-22. You need massive amounts of annotated data to train an accurate NER model, but getting that data requires hundreds of hours from domain experts who are already in short supply.

While general Biomedical NER (BMNER) models like BioBERT exist, they often fail to capture the specific nuances of nanobiology—a field that sits at the intersection of physics, chemistry, and biology. The authors sought to bridge this gap by developing a workflow that leverages existing structured knowledge (ontologies) to train a model that can "read" nanobiology papers as well as a human expert, without the prohibitive cost of manual annotation.

Methods

The researchers employed a three-step strategy to build NanoNER:

  1. Expert-in-the-Loop Ontology Curation: Instead of having experts label thousands of sentences, they had experts refine existing ontologies (like the Nanoparticle Ontology and eNanoMapper). This resulted in a curated vocabulary of 1,438 terms across five core labels: Nanoparticle, Property, Material, Event, and Technique.
  2. Distant Supervision: Using this curated vocabulary, they automatically annotated a large corpus of 728 full-text articles. This "distant supervision" approach treats the vocabulary as a silver standard to generate training data at scale.
  3. BioBERT Fine-tuning: They used BioBERT as the base architecture, fine-tuning it on the automatically labeled corpus to move beyond simple dictionary matching and toward true contextual understanding.

To prove the model wasn't just "memorizing" the dictionary, they performed ablation studies, where they intentionally hid 20% to 33% of the vocabulary during training to see if the model could correctly identify those "hidden" entities based on context alone.

Datasets

  • The Corpus: 728 full-text nanobiology articles, comprising over 158,000 sentences and 3.7 million tokens.
  • The Labels: The final dataset contained 121,201 entity occurrences.
  • Ontologies: The researchers harnessed the Nanoparticle Ontology (NPO) and eNanoMapper (ENM) as the primary knowledge sources.

Key Insights and Ideas

  • Context is King: The model demonstrated a "rediscovery" rate of up to 30%. This means that even when a specific term was completely removed from the training vocabulary, the model could often identify it as an entity simply by analyzing the surrounding linguistic context.
  • The "Anchor Word" Dependency: High-frequency terms (like the word "nanoparticle" itself) act as vital anchors. Removing these "top 10%" most frequent terms caused a significant drop in performance, suggesting the model relies on these common terms to orient itself within the domain's syntax.
  • Generic vs. Specific Language: The model performed better on Material and Property labels than on Event or Technique. This is because events and techniques often use "generic" scientific language (e.g., "microscopy" or "mitosis") that appears in many fields, leading to higher rates of confusion compared to highly specific terms like "gold nanoparticle."

Key Results

  • Overall Performance: On the full dataset, NanoNER achieved a near-perfect F1-score of 0.98.
  • Generalization: When tasked with finding "new" entities not present in the training set, the model maintained a precision between 0.77 and 0.81.
  • Ablation Success: In the 20% ablation tests, the model maintained a precision of ~0.79, proving it had learned the underlying patterns of the language rather than just a lookup table.

Limitations and Future Work

The authors acknowledge that the model is still heavily dependent on the quality and coverage of the initial resource (the ontology). If a concept isn't in the ontology or its variants, the distant supervision process might miss it, potentially leading the model to treat those instances as "negative" examples.

Future directions include: * Data Augmentation: Using techniques to balance the dataset (as some labels like "Material" were much more frequent than "Event"). * Knowledge Distillation: Implementing a teacher-student model (like the BOND framework) to iteratively improve the "silver" labels generated by distant supervision. * Cross-Domain Application: The authors believe this "low-manpower" workflow can be replicated for other niche scientific fields that lack large-scale annotated datasets.

Summary updated

Abstract

Here we present the training and evaluation of NanoNER, a Named Entity Recognition (NER) model for Nanobiology.NER consists in the identification of specific entities in spans of unstructured texts and is often a primary task in Natural Language Processing (NLP) and Information Extraction.The aim of our model is to recognise entities previously identified by domain experts as constituting the essential knowledge of the domain.Relying on ontologies, which provide us with a domain vocabulary and taxonomy, we implemented an iterative process enabling experts to determine the entities relevant to the domain at hand.We then delve into the potential of distant supervision learning in NER, supporting how this method can increase the quantity of annotated data with minimal additional manpower.On our full corpus of 728 full-text nanobiology articles, containing more than 120k entity occurrences, NanoNER obtained a F1-score of 0.98 on the recognition of previously known entities.Our model also demonstrated its ability to discover new entities in the text, with precision scores ranging from 0.77 to 0.81.Ablation experiments further confirmed this and allowed us to assess the dependency of our approach on the external resources.It highlighted the dependency of the approach to the resource, while also confirming its ability to rediscover up to 30% of the ablated terms.This paper details the methodology employed, experimental design, and key findings, providing valuable insights and directions for future related researches on NER in specialized domain.Furthermore, since our approach require minimal man-power, we believe that it can be generalized to other specialized fields.
BibTeX
@misc{cheng2024nanoner,
  author = {Ran Cheng and Martin Lentschat and Cyril Labbé},
  title = {{NanoNER: Named Entity Recognition for Nanobiology Using Experts’ Knowledge and Distant Supervision}},
  year = {2024},
  eprint = {2402.03362},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.03362},
}
Figure