The RST Continuity Corpus

Debopam Das, Markus EggACL 2023

TL;DR

This paper introduces the RST Continuity Corpus (RST-CC), a novel dataset annotated for seven dimensions of continuity (like time, space, and reference) within discourse relations. This resource aims to provide a more nuanced understanding of how discourse relations maintain or shift coherence, moving beyond binary continuous/discontinuous classifications.

What Problem Does This Paper Address?

Understanding how discourse relations contribute to the overall coherence of a text is crucial for natural language processing tasks like summarization, question answering, and dialogue systems. While discourse relations (e.g., cause, contrast, elaboration) are known to link segments of text, their role in maintaining or shifting the reader's "deictic center" – the point of reference for understanding the text – has been less systematically studied. Existing approaches often classify relations as simply "continuous" or "discontinuous," which is an oversimplification. Continuity can vary across different dimensions (e.g., a relation might be continuous in time but discontinuous in reference). This paper addresses the need for a detailed, dimension-specific analysis of continuity in discourse relations, which is currently lacking in corpus resources.

Methods

The authors introduce the RST Continuity Corpus (RST-CC), built upon the existing RST Discourse Treebank. The core innovation is the annotation of discourse relations with respect to seven continuity dimensions, operationalized from Givón's (1993) work:

  • Time: Categorized using Evers-Vermeul et al.'s (2017) classification of non-temporal, synchronous, chronological, and anti-chronological constellations. Anti-chronological pairs are considered discontinuous.
  • Space: Assessed based on whether events are non-spatial or situated in the same place (continuous), or if location shifts between segments (discontinuous).
  • Reference: Analyzed using Centering Theory principles, considering transitions like "continue," "retain," "smooth shift," and "rough shift" as continuous, while "zero" and "null" transitions are discontinuous.
  • Action: Evaluated based on script theory, determining if actions or events are part of a stereotypical routine (continuous) or not (discontinuous).
  • Perspective: Distinguished by objective, authorial, or other (quotation) perspectives. Continuity is maintained if segments share the same perspective.
  • Modality: Examined for shifts in the "modal source" or "possible world" dimension.
  • Speech Act: Assessed by changes in sentence mood (declarative, question, imperative).

The corpus also includes additional features like polarity and order of segments. Inter-annotator agreement was measured, with substantial agreement reported for most dimensions.

Datasets

The RST Continuity Corpus (RST-CC) is introduced in this paper. It comprises 1,009 relations from five major relation types (CAUSAL, CONTRASTIVE, CONDITIONAL, ELABORATION, TEMPORAL) selected from the RST Discourse Treebank (Carlson et al., 2002). The RST-CC is a novel benchmark for studying continuity in discourse relations.

Key Insights and Ideas

  • Continuity is multi-dimensional: The most significant insight is that discourse relations are not simply continuous or discontinuous; they exhibit varying degrees of continuity across different dimensions. This necessitates a fine-grained, dimension-specific analysis.
  • Binary classification is insufficient: The paper highlights how previous work's binary approach to continuity is problematic, as relations can be continuous in one dimension (e.g., time) and discontinuous in another (e.g., reference).
  • Operationalization is key: The authors demonstrate how abstract linguistic concepts like "continuity" can be operationalized into concrete, annotatable features, enabling corpus creation and systematic analysis.
  • Corpus limitations inform research: The authors acknowledge that the newspaper text domain of the RST-CC might skew results for dimensions like "action" and "speech act," suggesting the need for broader genre coverage in future work.

Key Results

  • The RST-CC contains 1,009 relations annotated for seven continuity dimensions.
  • Space and Speech Act: These dimensions show high uniformity across relation types, with relations being overwhelmingly continuous (>98% for space, >99% for speech act). This is attributed to the nature of newspaper texts.
  • Action: This dimension shows very low continuity (<2%), likely due to the non-narrative nature of the data.
  • Reference: Continuity scores for reference are consistently lower, possibly due to the prevalence of short segments in the RST-DT, which reduces the chance of finding shared referents.
  • Variability across relation types: Significant differences in continuity scores were observed across relation types for time, reference, perspective, and modality. For instance, CONTRASTIVE relations are least continuous for reference and perspective but highly continuous for time.

Limitations and Future Work

  • Genre bias: The current corpus is limited to newspaper texts, which may affect findings for dimensions like action and speech act. Future work will involve extending the corpus with more relation types and potentially other genres.
  • Fine-grained distinctions: While agreement was substantial, some distinctions (e.g., synchronous vs. chronological time) proved challenging and led to lower agreement, suggesting areas for refinement in annotation guidelines.
  • Broader scope: The authors plan to extend the RST-CC with more relation types (e.g., BACKGROUND, COMPARISON, EVALUATION, EXPLANATION) and make the final version publicly available via the Linguistic Data Consortium. This will enable more comprehensive analyses of discourse processing and signalling.

Summary updated

BibTeX
@inproceedings{das2023rst,
  author = {Debopam Das and Markus Egg},
  title = {{The RST Continuity Corpus}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure