Abstract
Joint extraction of entities and their relations benefits from the close interaction between named entities and their relation information. Therefore, how to effectively model such cross-modal interactions is critical for the final performance. Previous works have used simple methods such as label-feature concatenation to perform coarse-grained semantic fusion among cross-modal instances, but fail to capture fine-grained correlations over token and label spaces, resulting in insufficient interactions. In this paper, we propose a deep Cross-Modal Attention Network (CMAN) for joint entity and relation extraction. The network is carefully constructed by stacking multiple attention units in depth to fully model dense interactions over token-label spaces, in which two basic attention units are proposed to explicitly capture fine-grained correlations across different modalities (e.g., token-to-token and label-to-token). Experiment results on CoNLL04 dataset show that our model obtains state-of-the-art results by achieving 90.62% F1 on entity recognition and 72.97% F1 on relation classification. In ADE dataset, our model surpasses existing approaches by more than 1.9% F1 on relation classification. Extensive analyses further confirm the effectiveness of our approach.
Introduction
In Information Extraction, identifying entities (Named Entity Recognition, or NER) and determining their relationships (Relation Extraction, or RE) are deeply intertwined tasks. For example, knowing two words are "Locations" significantly increases the likelihood of a "Located In" relationship. However, many existing models treat these as separate steps or use simple feature concatenation, which fails to capture the nuanced, word-by-word correlations between specific tokens and their potential tags. Furthermore, standard RNN-based models often struggle with long-range dependencies, failing to link entities that appear at opposite ends of a long sentence.
Methods
The core innovation of this work is the Cross-Modal Attention Network (CMAN). The researchers frame joint extraction through the lens of multimodal learning, treating the sequence of words (tokens) and the sequence of entity tags (labels) as two distinct data modalities.
The architecture is built on two specialized components:
- BiLSTM-enhanced Self-Attention (BSA): This unit focuses on "intramodal" interactions, modeling how tokens relate to other tokens. It combines the sequential processing of a BiLSTM with the global reach of Multi-Head Attention.
- BiLSTM-enhanced Label-Attention (BLA): This "cross-modal" unit allows the model to explicitly attend to label information to refine token representations. During training, it uses ground-truth labels; during inference, it utilizes labels predicted by an upstream NER component.
These units are stacked in a deep, cascaded fashion, with the final output processed by a Conditional Random Field (CRF) for entity tagging and a multi-head mechanism for relation classification.
Key Results and Insights
The CMAN model established new state-of-the-art benchmarks on the CoNLL04 and ADE datasets. On CoNLL04, it achieved 90.62% F1 on entity recognition and 72.97% F1 on relation classification, outperforming previous models like SpERT. In the medical ADE dataset, it surpassed existing approaches by over 1.9% F1 on relation classification.
A significant finding is the model's robustness to "entity distance." While standard models often fail when entities are separated by many words, CMAN's attention mechanisms maintain high performance. In head-to-head comparisons for long-distance relations (>=20 tokens), CMAN's F1 score was nearly 14.67% higher than the baseline, demonstrating its ability to capture global dependencies that traditional RNNs miss.
Conclusion
By treating abstract tagging labels as a distinct modality and implementing dense cross-modal interactions, CMAN effectively bridges the gap between entity and relation extraction. While the model's RE performance is currently capped by the accuracy of the NER stage, the framework offers a promising direction for other NLP tasks where label-space information is underutilized, such as semantic role labeling or dependency parsing.
Summary updated