Abstract
This paper investigates two different neural architectures for the task of relation classification: convolutional neural networks and recurrent neural networks. For both models, we demonstrate the effect of different architectural choices. We present a new context representation for convolutional neural networks for relation classification (extended middle context). Furthermore, we propose connectionist bi-directional recurrent neural networks and introduce ranking loss for their optimization. Finally, we show that combining convolutional and recurrent neural networks using a simple voting scheme is accurate enough to improve results. Our neural models achieve state-of-the-art results on the SemEval 2010 relation classification task.
Introduction
Relation classification is a fundamental task in information extraction, aimed at identifying the semantic relationship between two marked entities within a sentence. For instance, in the sentence "The phone went into the washer", the goal is to identify the Entity-Destination relationship. Historically, state-of-the-art systems relied on Support Vector Machines (SVMs) paired with extensive, hand-crafted linguistic features. This paper explores neural architectures—specifically Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs)—to automate feature learning and improve classification accuracy.
Proposed Architectures
CNN with Extended Middle Context
Standard CNNs often struggle to balance the importance of the text between entities (the "middle" context) with the surrounding sentence. The authors introduce an Extended Middle Context approach. The sentence is divided into three regions: left of the first entity, between the entities, and right of the second entity. Two overlapping contexts are then formed: (Left + Entity 1 + Middle) and (Middle + Entity 2 + Right).
By repeating the middle segment in both inputs, the model naturally biases the max-pooling layer to prioritize the core relationship while still retaining peripheral information.
Connectionist Bi-directional RNN (C-BRNN)
While standard Bi-RNNs process sequences in both directions, they often rely solely on the final hidden states, which can lead to information loss for long sentences. The proposed Connectionist Bi-directional RNN (C-BRNN) addresses this by adding weighted connections from the previous combined hidden layer to the current one.
This architecture ensures that information from all intermediate steps is integrated into the final representation, preventing early context from being "washed out" by the end of the sequence.
Optimization via Ranking Loss
Instead of the conventional Softmax cross-entropy loss, the authors employ a ranking loss function. This objective maximizes the margin between the score of the correct relation and the score of the most competitive incorrect relation. This approach proved particularly effective for the multi-class nature of relation classification.
Experimental Results
The models were evaluated on the SemEval 2010 Task 8 dataset, which contains 10,717 sentences across 19 relation classes. The results demonstrate that both architectural innovations and the choice of loss function significantly contribute to performance.
| Model | F1 Score |
|---|---|
| Baseline CNN (50 dim) | 73.0 |
| ER-CNN (Extended Context + Ranking Loss) | 84.2 |
| R-RNN (Connectionist + Ranking Loss) | 83.4 |
| Ensemble (ER-CNN + R-RNN) | 84.9 |
The ensemble model, which combines the CNN and RNN via a simple voting scheme, achieved a new state-of-the-art F1 score of 84.9. This improvement highlights the complementary nature of the two architectures: CNNs excel at capturing local n-gram features, while RNNs are adept at modeling long-range dependencies across the entire sequence.
Conclusion
This work demonstrates that carefully designed neural architectures can outperform feature-heavy traditional models in relation classification. By introducing structural biases like the extended middle context and connectionist layers, and optimizing with ranking loss, the authors achieved state-of-the-art results without manual feature engineering. Future work may explore more complex ensembling techniques or the integration of lightweight linguistic features to further push performance boundaries.
Summary updated