Abstract
Relation classification (RC) identifies semantic relations between two entities within a given text. While current models perform well with abundant data, their performance degrades significantly in few-shot learning scenarios, especially when handling inverse relations. We propose the Function-words Adaptively Enhanced Attention (FAEA) framework, which employs a hybrid attention model to focus on class-related function words. To address the intra-class redundancy introduced by function words, we incorporate an adaptive message-passing mechanism to capture and transfer inter-class differences. Our results show that FAEA substantially improves performance on FewRel benchmarks, notably increasing inverse relation accuracy by 14.33% in 1-shot settings.
Introduction
Few-Shot Relation Classification (FSRC) aims to recognize novel relations using only a handful of labeled examples. A critical yet overlooked challenge in FSRC is the presence of inverse relations. In datasets like FewRel 1.0, over 21% of relations have an inverse counterpart (e.g., "A is part of B" vs. "B has part A").
Standard models typically focus on "content words" (nouns and verbs), which are often identical in inverse pairs. The semantic distinction in these cases is carried by "function words" like prepositions and auxiliary verbs. Current FSRC models often treat these as noise, leading to significant confusion between similar classes. The authors identify that while baseline models perform well on random tasks, their accuracy drops by nearly 27 points when forced to handle inverse relations in a 1-shot setting.
The FAEA Framework
The FAEA framework addresses the "inverse gap" through three primary components designed to elevate the importance of syntactic structure.
1. Instance Encoder
The model utilizes a standard BERT-based backbone to generate initial contextualized embeddings for the input sentence and the target entities. This provides a rich semantic foundation before the specialized attention layers are applied.
2. Function-words Enhanced Attention
This core component moves beyond simple keyword matching by implementing a hybrid attention strategy. Instead of a one-size-fits-all approach, FAEA uses:
- Class-general attention: This learns which function words (like "of", "in", "by") are globally significant across various relation types in the training set.
- Class-specific attention: This focuses on function words that are semantically or structurally related to the specific keywords of a relation. For example, it might highlight "of" specifically when the relation involves containment or part-whole structures.
3. Adaptive Message Passing (AMP)
While function words provide discriminative signals, they also introduce intra-class variance (redundancy) because the same function words appear in many different contexts. To counter this, FAEA treats instances as nodes in a graph. The AMP mechanism passes messages between nodes to reinforce similarities within a class and exaggerate differences between different classes. Mathematically, this acts as a filter to reduce the noise inherent in high-frequency function words while keeping the "signal" that defines the relation direction.
Experimental Results
The model was evaluated on FewRel 1.0, FewRel 2.0, and a specialized subset called FSIRC (Few-Shot Inverse Relation Classification), which specifically samples tasks containing inverse relations.
Performance on Standard Benchmarks
On FewRel 1.0 (5-way 1-shot), FAEA improved accuracy by 4.80% over strong baselines. When combined with pre-training (CP), it reached a state-of-the-art 94.11%. On FewRel 2.0, which tests domain adaptation (e.g., moving from general text to medical or chemical domains), the model showed superior robustness, outperforming BERT-Pair by over 6% in 1-shot tasks.
Inverse Relation Mastery
The most striking results came from the FSIRC task. FAEA outperformed the next best model (ConceptFERE) by 14.33% in the 1-shot setting. This confirms the hypothesis that function words are the primary syntactic markers that define the direction of a relationship in low-data regimes.
Conclusion
FAEA demonstrates that function words, often discarded as stop words, are vital structural signals for few-shot learning. By combining hybrid attention with adaptive message passing, the framework effectively balances the need for discriminative syntactic features with the requirement for low intra-class variance. These findings suggest that future FSRC models should move away from purely content-word-centric approaches and embrace the structural nuances of language.
Summary updated