Building and Refining Rhetorical-Semantic Relation Models

Sasha Blair-Goldensohn, Kathleen R. McKeown, Owen RambowNAACL 2007 · OpenReview

Abstract

This paper presents advancements in the automated detection of implicit discourse relations, such as "Cause" or "Contrast," by refining the established "cue-phrase stripping" methodology. Through systematic parameter optimization and the application of topic segmentation and syntactic parsing to filter noisy training data, the authors demonstrate significant performance improvements on both synthetic benchmarks and human-annotated datasets like the Penn Discourse TreeBank (PDTB).

Introduction

Rhetorical-semantic relations (RSRs) are fundamental to text coherence. While explicit relations are signaled by cue phrases (e.g., "because," "however"), implicit relations—where the connection is inferred by the reader—pose a significant challenge for natural language processing. This work builds upon the framework introduced by Marcu and Echihabi (2002), which utilizes large-scale synthetic data generated by stripping cue phrases from explicit relations to train classifiers for implicit ones.

Methodology

The authors developed TextRels, a platform extending the Marcu and Echihabi (M&E) framework. The system employs a Naive Bayesian classifier that analyzes the Cartesian product of word pairs across text spans to determine the most likely relation.

Data Extraction

To address the inherent noise in synthetic data, the authors mined millions of instances from the Gigaword corpus using specific linguistic patterns to identify Cause, Contrast, and NoRel (no relation) examples.

Table 1: Sample patterns and instance counts for mined training data.

Refinements

The paper introduces three primary refinements to the baseline model:

  1. Parameter Optimization: Systematic tuning of hyperparameters, including Laplace smoothing (setting $\lambda$ to 0.25), vocabulary size (6,400 stems), and the impact of stoplists. Notably, the authors found that removing stop words was counter-productive, as high-frequency words carry subtle discourse signals.
  2. Topic Segmentation: Utilizing LCSeg to ensure that "Cause" and "Contrast" instances do not span major topic boundaries. Conversely, these boundaries were used to better define "NoRel" examples.
  3. Syntactic Heuristics: Employing the Collins parser to isolate the specific clauses participating in a relation. This filtering trims extraneous text that does not contribute to the rhetorical link.

Experimental Results

Evaluation was conducted on both synthetic test sets (Auto/Auto-S) and the human-annotated PDTB, which represents "organic" implicit relations.

Key Findings

  • Optimization and Segmentation: Tuning parameters and applying topic segmentation yielded consistent gains. For example, on the synthetic "Auto" test set, Contrast vs. NoRel accuracy rose from 70.7% to 71.3%.
  • Performance Comparison: The refined models outperformed the original M&E baseline across various classification tasks, as shown in the table below.
Table 2: Classification accuracy comparison across different test sets and model configurations.

On the difficult PDTB dataset, the optimized classifier achieved 75.2% accuracy for Cause vs. NoRel and 61.1% for Cause vs. Contrast. While syntactic filtering provided a 2.7% boost for Cause vs. NoRel on PDTB, the limited size of the human-annotated set made it difficult to claim broad statistical significance for that specific feature.

Conclusion

The study demonstrates that the "synthetic" data approach for implicit relation detection can be significantly improved through structural filtering and careful parameter tuning. However, a performance gap remains between synthetic and "organic" data, indicating that implicit relations in natural text may possess unique characteristics not fully captured by stripped explicit markers. Future work will focus on expanding syntactic heuristics to cross-sentence boundaries and modeling the predictability of relations to better bridge this gap.

Summary updated

Abstract

Sasha Blair-Goldensohn, Kathleen McKeown, Owen Rambow. Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference. 2007.
BibTeX
@inproceedings{blairgoldensohn2007building,
  author = {Sasha Blair-Goldensohn and Kathleen R. McKeown and Owen Rambow},
  title = {{Building and Refining Rhetorical-Semantic Relation Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
  url = {https://openreview.net/forum?id=ryNF1Xb_-r},
}
Figure