Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese

Sheng Xu, Peifeng Li, Fang Kong, Qiaoming Zhu, Guodong ZhouACL 2019 · OpenReview

Abstract

In the literature, most of the previous studies on English implicit discourse relation recognition only use sentence-level representations, which cannot provide enough semantic information in Chinese due to its unique paratactic characteristics. In this paper, we propose a topic tensor network to recognize Chinese implicit discourse relations with both sentence-level and topic-level representations. In particular, besides encoding arguments (discourse units) using a gated convolutional network to obtain sentence-level representations, we train a simplified topic model to infer the latent topic-level representations. Moreover, we feed the two pairs of representations to two factored tensor networks, respectively, to capture both the sentence-level interactions and topic-level relevance using multi-slice tensors. Experimentation on CDTB, a Chinese discourse corpus, shows that our proposed model significantly outperforms several state-of-the-art baselines in both micro and macro F1-scores.

Introduction

Recognizing implicit discourse relations—where connecting words like "because" or "but" are omitted—is a significant challenge in natural language processing. This task is particularly difficult for the Chinese language, which is characterized as paratactic. Unlike English (hypotactic), which relies on formal grammatical cohesion, Chinese relies heavily on semantic flow. In the Chinese Discourse Treebank (CDTB), 75.2% of relations are implicit, compared to approximately 40% in the English Penn Discourse Treebank (PDTB).

Existing models often rely solely on sentence-level representations, which may fail to capture the broader topical context necessary for linking segments in a paratactic language. This paper introduces the Topic Tensor Network (TTN), a model that integrates sentence-level deep learning with latent topic modeling to capture both local and global semantic interactions.

Methodology

The TTN architecture consists of three primary components designed to handle the complexities of Chinese discourse:

  1. GCN-based Encoder: A Gated Convolutional Network (GCN) extracts sentence-level features. GCNs are effective at capturing hierarchical features from long text segments while maintaining computational efficiency.
  2. Simplified Topic Model (STM): To capture broader context, a Variational Autoencoder (VAE)-based topic model infers latent topic distributions from a Bag-of-Words representation of the arguments. This "simplified" model focuses on word distribution restoration rather than full document reconstruction, making it robust for smaller datasets.
  3. Factored Tensor Network (FTN): To model the interaction between arguments, the authors employ a tensor network. By using tensor factorization (low-rank approximation), the model significantly reduces parameter complexity (by approximately 20x compared to standard Neural Tensor Networks), allowing for more "slices" to capture complex semantic interactions without overfitting.
The architecture of the proposed Topic Tensor Network (TTN), featuring the Simplified Topic Model and Factored Tensor Networks.

The model is trained via joint learning, optimizing for both discourse relation classification and topic reconstruction accuracy simultaneously.

Experimental Setup

The model was evaluated on the Chinese Discourse Treebank (CDTB), which contains 500 newswire documents. The study focused on four top-level discourse classes: Causality (Cause-Effect), Coordination (Parallel ideas), Elaboration (Adding detail), and Transition (Contrast/Change).

Results

The TTN model achieved state-of-the-art results, outperforming several strong baselines, including Bi-LSTMs, CNNs, and previous neural tensor models.

Performance comparison of the TTN model against baseline models on the CDTB dataset.

Key performance highlights include:

  • Overall Performance: The TTN achieved a 73.6% Micro-F1 and 67.8% Macro-F1, a significant improvement over previous state-of-the-art models which typically reached 70-71% Micro-F1.
  • Elaboration Boost: The inclusion of the STM led to a substantial +8.6 F1 score improvement specifically for the Elaboration relation, demonstrating the value of topical context.
  • SOTA Comparison: The model outperformed the previous best model (Guo et al., 2018) by 2.9 points in Micro-F1 and 2.0 points in Macro-F1.

Discussion and Insights

The research highlights that topic continuity is a powerful signal for relations like Elaboration. Knowing that two arguments share a latent topic (e.g., "economics") is often more informative than syntactic analysis alone.

However, a "Causality Trade-off" was observed: adding topic information slightly decreased performance on the Causality relation. Error analysis showed that causal arguments often share the same topic, leading the model to misclassify them as Coordination (51.3% of Causality errors). This suggests that logical connections do not always align perfectly with topical similarity.

Conclusion

The Topic Tensor Network successfully addresses the challenges of implicit discourse recognition in Chinese by combining sentence-level and topic-level representations. By utilizing factored tensor networks, the model achieves high performance with reduced parameter complexity, setting a new benchmark for the CDTB corpus. Future work may explore specialized representations for specific relations to further mitigate misclassifications between topically similar categories.

Summary updated

Abstract

In the literature, most of the previous studies on English implicit discourse relation recognition only use sentence-level representations, which cannot provide enough semantic information in Chinese due to its unique paratactic characteristics. In this paper, we propose a topic tensor network to recognize Chinese implicit discourse relations with both sentence-level and topic-level representations. In particular, besides encoding arguments (discourse units) using a gated convolutional network to obtain sentence-level representations, we train a simplified topic model to infer the latent topic-level representations. Moreover, we feed the two pairs of representations to two factored tensor networks, respectively, to capture both the sentence-level interactions and topic-level relevance using multi-slice tensors. Experimentation on CDTB, a Chinese discourse corpus, shows that our proposed model significantly outperforms several state-of-the-art baselines in both micro and macro F1-scores.
BibTeX
@inproceedings{xu2019topic,
  author = {Sheng Xu and Peifeng Li and Fang Kong and Qiaoming Zhu and Guodong Zhou},
  title = {{Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=VapyNhrnUc},
}
Figure