Introduction
Implicit Discourse Relation Recognition (IDRR) is a critical task in natural language processing that involves identifying the semantic relationship between two text segments (arguments) when explicit discourse connectives (e.g., "because", "but", "however") are absent. For instance, in the sequence "I forgot my umbrella. I got soaked.", a human reader easily infers a Contingency (Cause) relationship. However, for computational models, the lack of explicit markers makes this a challenging inference problem.
Existing approaches often struggle because different discourse relations are signaled by different types of linguistic cues. Some relations depend on core semantic content (nouns and verbs), while others are hinted at by rhetorical devices or function words. This paper introduces the Multi-Attentive Neural Fusion (MANF) model, which addresses these challenges by simultaneously capturing contextual semantic connections and fine-grained linguistic evidence.
The Multi-Attentive Neural Fusion (MANF) Framework
The MANF model is designed to integrate two distinct types of information through a hierarchical architecture.
Dual Attention Network (DAN)
The DAN module processes the two arguments using a BiLSTM to capture sequential dependencies. It then applies two complementary attention mechanisms:
- Self-Attention: This focuses on the internal context of each argument, typically highlighting "content words" that define the core topic.
- Interactive Attention: This mechanism weights words in one argument based on their relevance to the other argument. This often captures "function words" or rhetorical signals that bridge the two segments.
These two representations are combined via a fusion gate to produce a "semantic connection" representation ($v_d$).
Offset Matrix Network (OMN)
A key innovation of this work is the Offset Matrix Network. It leverages the linear translation property of word embeddings (e.g., $vec(king) - vec(man) \approx vec(queen) - vec(woman)$). The OMN calculates the difference vector (offset) for every possible word pair across the two arguments. This captures latent linguistic evidence—such as semantic regularity, antonymy, or causality—that exists independently of the specific sentence context. The resulting matrix is processed through a multilayer perceptron to generate a "linguistic evidence" representation ($v_o$).
Hierarchical Fusion
Finally, a hierarchical fusion mechanism uses sigmoid-based gates to adaptively combine $v_d$ and $v_o$. This allows the model to dynamically prioritize either the broad semantic context or the fine-grained word-pair evidence depending on the specific relation being classified.
Key Insights: The "Two Lenses" Theory
The authors demonstrate that self-attention and interactive attention act as "two lenses" that view discourse relations differently.
As shown in the visualization above, different relations exhibit distinct attention patterns:
- Expansion: Often relies on content words, where self-attention provides strong signals.
- Comparison and Temporal: These relations frequently show higher activation in interactive attention, as the model looks for contrastive or sequential markers across the argument boundary.
Experimental Results
The MANF model was evaluated on the Penn Discourse TreeBank (PDTB) 3.0, the current gold standard for discourse analysis. The model achieved state-of-the-art performance:
- Macro $F_1$ Score: Reached 53.14% with word2vec and 56.63% with BERT, outperforming previous benchmarks like IPAL and NNMA.
- Category Performance: The model showed significant improvements in "Comparison" and "Temporal" categories, which are historically the most difficult for IDRR systems.
- Ablation Studies: Removing either the DAN or the OMN resulted in a measurable performance drop, confirming that both contextual semantics and word-pair linguistic offsets are essential for robust recognition.
Conclusion
The MANF model advances the state-of-the-art in implicit discourse recognition by bridging the gap between high-level semantic context and low-level linguistic evidence. By utilizing a dual-attention mechanism and a novel Offset Matrix Network, the model effectively mimics the human ability to infer hidden relationships in text. Future research may explore the integration of larger language models and more complex fusion strategies to further refine these inferences.
Summary updated