TL;DR
This paper introduces an adversarial learning approach to improve discourse rhetorical structure parsing, a task notoriously limited by data scarcity. By transforming discourse trees into visual representations and using a Generative Adversarial Network (GAN), the model learns to optimize the entire tree structure globally, leading to state-of-the-art results on benchmark datasets.
What Problem Does This Paper Address?
Discourse Rhetorical Structure (DRS) parsing aims to represent the hierarchical organization of text, identifying how different parts of a text relate to each other (e.g., cause, elaboration, contrast). This is crucial for understanding text structure and has downstream applications in summarization, question answering, and more. However, a major challenge in this field is the lack of large, annotated datasets. Existing state-of-the-art parsers often make local decisions at each step, which can lead to suboptimal global structures. This paper addresses this limitation by proposing a method for global optimization of DRS parsing, aiming to improve the overall coherence and accuracy of the generated discourse trees.
Methods
The core innovation of this paper lies in its adversarial learning framework for global optimization. The authors first transform both gold-standard and predicted DRS trees into a visual, image-like representation using two color channels: one for structure (ST) and another for nuclearity-relation (NR) tags. This graphical representation allows the model to "see" the entire tree structure.
They then employ a Least Squares Generative Adversarial Network (LSGAN) to learn from these visual representations. The LSGAN consists of two main components:
- A Generative Net (G): This is essentially the DRS parser itself, which aims to generate "fake" DRS tree representations.
- A Discriminative Net (D): This acts as an "adversary" that tries to distinguish between real (gold-standard) DRS tree representations and fake ones generated by G.
The adversarial training process encourages the parser (G) to generate more realistic DRS trees that can fool the discriminator (D). This global optimization objective complements the traditional local training objectives of the parser, leading to improved overall performance. The authors also simplify the baseline top-down parser by removing dummy split points and using a combined nuclearity-relation (N-R) tag prediction, reducing the number of classifiers.
Datasets
The authors evaluate their approach on two widely used benchmark datasets for discourse rhetorical structure parsing:
- RST-DT (English RST Discourse Treebank): This corpus contains 385 Wall Street Journal articles.
- CDTB (Chinese Connective-driven Discourse Tree-Bank): This corpus consists of 500 newswire articles, segmented into paragraphs and EDUs.
Key Insights and Ideas
- Visualizing Discourse Trees for Adversarial Learning: Transforming DRS trees into a 2D graphical representation is a novel way to enable adversarial learning for this task, allowing the model to learn from the global structure.
- LSGAN for Stable Adversarial Training: The use of Least Squares GAN (LSGAN) is highlighted as a method to improve the stability of adversarial training, mitigating issues like gradient vanishing that can plague traditional GANs.
- Global Optimization Improves Local Decisions: The experimental results suggest that optimizing for the entire tree structure (global optimization) can indeed lead to better local decisions within the parsing process.
- Joint N-R Prediction is Beneficial: Combining nuclearity and relation prediction into a single N-R tag prediction task proves effective, especially when dealing with limited data.
Key Results
The proposed adversarial learning method achieves state-of-the-art results on both the RST-DT and CDTB corpora.
- On the RST-DT corpus, the ELMO-based parser with adversarial learning shows significant improvements, particularly in structure and nuclearity prediction, outperforming previous state-of-the-art parsers.
- On the CDTB corpus, the adversarial approach also leads to comparable or better performance across all indicators, demonstrating its effectiveness for Chinese discourse parsing.
- The paper also shows that using powerful language models like XLNet further boosts performance, suggesting the potential for integrating advanced pre-trained models with this adversarial framework.
Limitations and Future Work
The authors acknowledge that while their adversarial approach is effective, the improvements are not always "qualitative" and there is "much room for improvement." They suggest that the instability of adversarial learning processes is an ongoing challenge. Future work could explore more advanced adversarial techniques or different graphical representations to further enhance the global optimization of DRS parsing. The paper also notes that the Chinese rhetorical relation prediction might be facing a bottleneck due to data limitations, suggesting this area warrants further investigation.
Summary updated