Scaling Discourse Parsing through Weak Supervision and Cross-lingual Projection

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2109 papers119 conceptsfrozen 2026-10-04

While modern Natural Language Processing (NLP) has mastered the art of understanding individual sentences, the "logic" that binds those sentences into a coherent document—known as discourse structure—remains a significant challenge. The primary obstacle is not a lack of algorithms, but a severe data bottleneck. Unlike part-of-speech tagging or sentiment analysis, which can often be crowdsourced, identifying rhetorical relations (like Contrast, Causality, or Elaboration) requires specialized linguistic expertise and an understanding of global document context.

This scarcity of "gold-standard" data has led researchers to develop creative strategies to manufacture training signals where none exist, effectively scaling discourse-aware models without the need for exhaustive manual annotation.

The Discourse Data Bottleneck

Most deep learning architectures are "data-hungry," requiring massive datasets to generalize. However, as Gregor Weiss notes in Learning Representations for Text-level Discourse Parsing, the high cost of expert annotation for complex relational structures results in datasets that are often too small for robust neural training. This is particularly acute for "implicit" relations—logical links that aren't signaled by explicit words like "because" or "but."

Shafiq Joty and colleagues emphasize that this dependency on expert knowledge prevents discourse analysis from scaling at the same rate as other NLP tasks (Discourse Analysis and Its Applications). To bridge this gap, the field has shifted toward "silver-standard" data: labels that are automatically inferred rather than manually verified.

Distant Supervision: Sentiment as a Proxy

One of the most effective ways to generate discourse labels at scale is to use other, more abundant signals as a proxy. Patrick Huber and Giuseppe Carenini pioneered this with MEGA-DT, a dataset of 250,000 documents (MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision).

The intuition is simple: the rhetorical structure of a review often mirrors its sentiment flow. By using sentiment analysis as a "distant" supervisor, they could infer the hierarchy and importance (nuclearity) of different text segments. Similarly, Parminder Bhatia and Jacob Eisenstein showed that weighting "important" parts of a discourse tree more heavily leads to better document-level sentiment analysis, creating a virtuous cycle between the two tasks (Better Document-level Sentiment Analysis from RST Discourse Parsing).

Cross-lingual Projection: Triangulating Meaning

Another powerful mechanism for overcoming data scarcity is "cross-lingual explicitation." Languages differ in how often they use explicit connecting words. For instance, a logical connection that is "implicit" in English might be "explicit" in a French or German translation.

Wei Shi and Vera Demberg leveraged this by looking for instances where translators added words like "because" to clarify an English sentence pair (Acquiring Annotated Data with Cross-lingual Explicitation for Implicit Discourse Relation Classification). By "triangulating" across multiple languages, they can automatically label English data with high precision. This approach has been extended to low-resource settings, such as Marian Marchal’s work on Nigerian Pidgin, where annotation projection helps capture logical links in languages that lack large-scale treebanks (DiscoNaija: a discourse-annotated parallel Nigerian Pidgin-English corpus).

Multi-task Learning and Structural Priors

When labeled data is scarce, researchers often force models to learn discourse structure as a "side effect" of solving a different problem. This is known as multi-task learning.

The LLM Era: Incremental and Synthetic Parsing

The most recent shift involves using Large Language Models (LLMs) to either generate synthetic training data or perform parsing incrementally. Kate Thompson and Nicholas Asher developed Llamipa, which treats discourse parsing as a step-by-step conversation, using the previously inferred graph as context for the next prediction (Llamipa: An Incremental Discourse Parser).

Meanwhile, in extremely low-resource settings, researchers like Agniva Maiti are using "human-in-the-loop" synthetic pipelines, where LLMs generate initial datasets that are then refined by native speakers, drastically reducing the time needed to build linguistic toolkits from scratch (NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Data).

By combining these diverse strategies—distant supervision, cross-lingual projection, and multi-task learning—the research community is successfully moving past the data bottleneck, enabling machines to understand not just what a text says, but how its ideas are logically connected.

Papers in this version

Showing 60 of 109 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 135 papers
  2. Version 1 · 2026-10-04 · 109 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_discourse_data_scarcity_53e7fb00,
  title = {discourse data scarcity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/discourse-data-scarcity/rdr_53e7fb00d5dc481e894896bdb9d14e79},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_scaling_discourse_parsing_through_weak_s_53e7fb00_v1,
  title = {Scaling Discourse Parsing through Weak Supervision and Cross-lingual Projection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scaling-discourse-parsing-through-weak-supervision-and-cross-lingual-projection/rdr_53e7fb00d5dc481e894896bdb9d14e79/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}