Abstract
The AMFF (Attention-based Multi-level Feature Fusion) framework improves Named Entity Recognition (NER) by simultaneously capturing local and global information at both the character and word levels. By using attention mechanisms to weight these diverse perspectives, the model excels at disambiguating polysemous words—like "Washington"—that can represent different entity types depending on the broader context.
Problem Statement: The Challenge of Polysemy
Named Entity Recognition is a cornerstone of Natural Language Processing, yet it frequently stumbles on "polysemy"—words that have multiple meanings or categories. For instance, "Washington" could be a person (George Washington), a location (Washington State), or part of an organization (Washington University).
Traditional models often rely on local context windows or simple character-level embeddings. While effective, these methods often miss "global" cues hidden elsewhere in a sentence or fail to integrate specific syntactic information like capitalization and lexical phrases. The authors argue that a truly robust NER system needs to look at the text through multiple lenses simultaneously: from the fine-grained shapes of characters to the broad relationships between distant words.
Methodology: The AMFF Framework
The authors introduce the AMFF framework, which processes input text through four parallel "feature selection" components:
- Global Character-level: Uses a BiLSTM combined with self-attention to capture long-distance dependencies between characters across the entire sequence.
- Local Character-level: Employs a CNN with max-pooling to extract morphological features, such as capitalization, which are strong indicators for entities.
- Global Word-level: Utilizes self-attention on word embeddings to identify lexical phrases and keywords (e.g., recognizing that the word "in" often precedes a location).
- Local Word-level: Distills the most prominent features from the word-level attention output using max-pooling and a feed-forward network.
Once these four feature sets are extracted, the model uses an automatic fusion strategy. It assigns a learnable weight ($\lambda$) to each component, allowing the model to decide which "lens" is most important for a given dataset. Finally, these fused features are fed into a standard BiLSTM-CRF (Conditional Random Field) layer to predict the final sequence of entity tags.
Experimental Setup and Datasets
The model was evaluated on four diverse benchmark datasets to test its generalizability across different domains:
- CoNLL-2003: A standard benchmark for general-purpose NER (News).
- NCBI-disease: A medical dataset focused on identifying disease names.
- SciERC: A dataset derived from scientific AI paper abstracts.
- JNLPBA: A technical bio-entity dataset.
Key Insights and Results
The AMFF model consistently outperformed state-of-the-art baselines across all four datasets. It achieved an F1 score of 94.48% on CoNLL-2003 and 92.11% on NCBI-disease.
A key "aha moment" of this research is demonstrating that global attention can solve local ambiguity. In the sentence "Washington University... is located in Missouri," the word "University" might be far from "Washington," but the global attention mechanism links them, correctly identifying "Washington" as part of an Organization (ORG) rather than a Person (PER). Through ablation studies, the researchers also found that word-level features generally contributed more to the final score than character-level features, likely because pre-trained word embeddings already carry significant semantic weight.
Conclusion and Future Work
While the AMFF framework is powerful, the authors note that the architecture is designed to be extensible. Future work could involve adding specific linguistic features like affixes or part-of-speech tags. Additionally, integrating external knowledge bases (like Wikipedia or domain-specific ontologies) could further improve performance on highly technical or rare entities. While the parallel feature extraction is effective, the trade-off between this added complexity and inference speed remains an area for further exploration.
Summary updated