Abstract
Recognizing implicit discourse relations—understanding logical connections between sentences without explicit connectives—is a fundamental challenge in natural language processing. This paper introduces a novel TransS-driven joint learning architecture that models discourse relations as geometric translations in a vector space ($Arg1 + Relation \approx Arg2$). By integrating Geometric Structure Learning (GSL) with Semantic Feature Learning (SFL), the model captures latent logical directions more effectively than traditional semantic-only methods. Evaluated on the Penn Discourse TreeBank (PDTB) 2.0, the architecture achieves state-of-the-art performance, particularly in the challenging Comparison category, demonstrating the efficacy of geometric constraints in discourse analysis.
Introduction
Discourse relations describe the logical links between text units. While explicit relations are easily identified by connectives like "because" or "but," implicit discourse relation recognition (IDRR) requires the model to infer these links from the context of two arguments (Arg1 and Arg2).
Current deep learning approaches typically focus on encoding arguments or identifying word-level interactions. However, these methods often fail to exploit the geometric structure of the embedding space or the explicit signals provided by relation labels. The authors argue that a discourse relation is not just a category but a transformation that 'bridges' the thoughts expressed in the two arguments.
Methodology
The proposed architecture consists of four primary components designed to capture both semantic and structural information.
1. Multi-level Encoder
The model uses a multi-level encoder to produce dense representations of Arg1 and Arg2. This module employs stacked BiLSTM layers combined with an attention mechanism to focus on the most relevant parts of each argument.
2. Geometric Structure Learning (GSL) Module
The core innovation of this work is the GSL module, inspired by the TransE model for Knowledge Graph completion. It treats the discourse relation as a translation vector $r_s$. The model aims to satisfy the algebraic equation $h_s + r_s \approx t_s$, where $h_s$ and $t_s$ are the head (Arg1) and tail (Arg2) representations. This geometric constraint forces the model to learn the underlying 'direction' of the logic connecting the two sentences.
3. Semantic Feature Learning (SFL) Module
The SFL module acts as a standard classifier, using a softmax layer to predict the relation label based on the argument embeddings. This ensures that the representations remain semantically rich and discriminative for classification.
4. Joint Learning and Contextual Embeddings
The model is trained using a joint loss function that balances geometric and semantic objectives. To further enhance performance, the authors incorporate ELMo (Embeddings from Language Models) for contextualized word representations, providing a stronger foundation for the encoders.
Experimental Evaluation
The model was tested on the Penn Discourse TreeBank (PDTB) 2.0 using the standard 'Rutherford' split. The evaluation focused on Level-1 classification (Comparison, Contingency, Expansion, and Temporal).
Key Results
- State-of-the-Art Performance: The TransS-driven model achieved a macro-average F1 score of 51.24% and an accuracy of 59.94%, outperforming previous joint learning baselines.
- Improvement in Comparison: The model showed a significant jump in the Comparison category, reaching an F1 of 47.98%, compared to approximately 40% in earlier models.
- Ablation Study: Removing the GSL module resulted in a performance drop to 46.46% F1, proving that the geometric translation approach is a critical driver of the model's success.
Conclusion
The TransS-driven joint learning architecture successfully demonstrates that modeling discourse relations as geometric transformations can significantly improve recognition accuracy. By combining geometric constraints with semantic features, the model captures the latent logic of human discourse more effectively. Future research may explore applying these learned relation transformations to other high-level NLP tasks such as text summarization or machine translation.
Summary updated