Multi-Channel Convolutional Neural Networks with Adversarial Training for Few-Shot Relation Classification (Student Abstract)

Yuxiang Xie, Hua Xu, Congcong Yang, Kai Gao

Relation Classification (RC) is a fundamental task in information extraction, aimed at identifying semantic relationships between entities in text. While deep learning has significantly advanced RC, traditional models require vast amounts of labeled data. This research explores Few-Shot Relation Classification, where a model must learn to recognize new relations from only a handful of examples.

To address the primary challenge of few-shot learning—overfitting on small, potentially noisy datasets—the authors propose a novel architecture combining multi-channel convolutional features with adversarial training.

Methodology

The proposed model, Proto-Mul+AT, builds upon the Prototypical Network framework. It introduces two key technical innovations to improve feature extraction and model robustness.

Multi-Channel Convolutional Neural Network (Proto-Mul)

In standard NLP models, word embeddings and position embeddings (the distance of a word from target entities) are typically concatenated into a single vector. This research instead treats them as separate input channels, analogous to RGB channels in image processing.

  • Channel 1: Word embeddings.
  • Channels 2 & 3: Position embeddings relative to the two entities.

By using multi-channel convolution, the model can simultaneously extract local features across these distinct spatial representations, capturing more nuanced interactions between a word's meaning and its structural context.

Architecture of the Multi-Channel CNN with adversarial perturbations.

Adversarial Training (AT)

To prevent the model from overfitting to the specific noise of a few-shot sample, the authors employ adversarial training. During the training phase, small perturbations ($e^m$ in the figure above) are added to the input embeddings. These perturbations are calculated to maximize the model's loss, effectively "attacking" the model with difficult examples. By learning to classify these adversarial inputs correctly, the model develops more stable and generalized class prototypes.

Experimental Evaluation

The model was evaluated on the FewRel dataset, a standard benchmark containing 100 relations with 700 instances each. The researchers tested the model in "K-way J-shot" scenarios, specifically 5-way 5-shot and 10-way 5-shot tasks.

Performance comparison on the FewRel dataset showing the superiority of the Proto-Mul+AT approach.

As shown in the results table, the Proto-Mul+AT model consistently outperforms the standard Proto-CNN baseline by approximately 6–7%. Notably, it also achieves better performance than Proto-HATT, a more complex model utilizing hybrid attention mechanisms. This suggests that strengthening the fundamental feature extractor and regularizing via adversarial noise can be more effective than increasing architectural complexity.

Conclusion

This research demonstrates that treating different embedding types as separate channels allows CNNs to learn more robust structural features for relation classification. Furthermore, adversarial training serves as a powerful regularizer in few-shot scenarios, smoothing decision boundaries and creating more representative class prototypes. Future work may explore the application of these multi-channel adversarial techniques to other low-resource NLP tasks.

Summary updated

Abstract

The distant supervised (DS) method has improved the performance of relation classification (RC) by means of extending the dataset. However, DS also brings the problem of wrong labeling. Contrary to DS, the few-shot method relies on few supervised data to predict the unseen classes. In this paper, we use word embedding and position embedding to construct multi-channel vector representation and use the multi-channel convolutional method to extract features of sentences. Moreover, in order to alleviate few-shot learning to be sensitive to overfitting, we introduce adversarial learning for training a robust model. Experiments on the FewRel dataset show that our model achieves significant and consistent improvements on few-shot RC as compared with baselines.
BibTeX
@inproceedings{xie2020multi,
  author = {Yuxiang Xie and Hua Xu and Congcong Yang and Kai Gao},
  title = {{Multi-Channel Convolutional Neural Networks with Adversarial Training for Few-Shot Relation Classification (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7256},
}
Figure