Discourse parsers tell how two neighbouring sentences relate: one causes the other, contrasts with it, or follows it in time. When the relation is "implicit", meaning no word like "because" or "however" marks it, these parsers break down outside the newspaper text they were trained on. A model trained on Wall Street Journal articles loses about 31 points of accuracy on other genres. The standard fix for this kind of train/test mismatch is adversarial domain adaptation: a second network tries to guess which genre a representation came from, and the main encoder is trained to fool it, so its features look the same across genres. Our claim is that this fix is built on an assumption that discourse data violates. Genres do not just phrase relations differently. They use different relations at different rates. News is dominated by elaboration, while talks and narratives lean on temporal and causal links. If you force features from such genres to look identical, you also force the model to predict the source genre's mix of relations.
We propose to estimate how often each relation occurs in the unlabeled target genre and to correct for that difference before aligning. The estimate comes from a signal that survives genre shift: explicit connectives. These can be detected in raw target text with high reliability and they reveal which relations the genre favours. The resulting method reweights a class-conditional adversarial objective, so alignment matches "what a Cause looks like" across genres rather than "what the average relation looks like". Prior-correction methods for this kind of frequency difference, usually called label shift, exist in general machine learning, but nothing in our research file applies them to discourse. The open question is whether connectives can supply the frequency estimate without any target labels.
Core Claim
Cross-genre transfer of implicit relations is mostly a problem of shifted relation frequencies, not only shifted wording. Under that kind of shift, standard adversarial alignment is expected to transfer worse as the gap in relation frequencies grows. Prior-corrected conditional alignment should not degrade this way. The target frequencies can be estimated from explicit connectives in unlabeled target text, through a ratio between explicit and implicit relation rates that is measured on the source corpus and assumed to carry over to new genres. That assumption can be checked directly on the GDTB treebank, which annotates both explicit and implicit relations.
Grounding in Prior Work
Three papers motivate the gap. GDTB converts the multi-genre GUM corpus into the Penn Discourse Treebank format across 16 genres. It reports that a model trained only on PDTB reaches 44.57% accuracy on these genres, and that supervised joint training raises this to 64.40%. It also finds implicit relations and spoken genres the hardest. DiscoGeM crowdsources implicit relations in European Parliament proceedings, novels and Wikipedia. It finds that about 31% of expert-annotated items admit several valid labels. Discourse Sense Flows shows that sequences of relation senses identify a document's genre at roughly 0.70 macro F1. Together these show that the label distribution itself is a genre fingerprint. The same paper's connective tagger reaches 93.65 F1 on PDTB, which is the kind of tool we rely on for estimating target frequencies.
Two papers supply our method. Conditional Adversarial Domain Adaptation, known as CDAN, argues that aligning feature distributions without regard to class can pair up the wrong classes across domains. It conditions the genre-guessing network on the outer product of features and predicted class probabilities, and it adds a weighting that favours confident examples (the CDAN+E variant). CDAN still implicitly treats source and target class frequencies as comparable. Our proposal adds the correction for unequal frequencies, which is the case discourse genres present. Genre Separation Network splits each input into a genre-specific part and a shared part, and adds a reconstruction loss. It gained 1.2 to 1.7 F1 over a standard adversarial baseline on cross-genre entity relation extraction. We borrow its reconstruction term as a guard against the adversarial objective erasing useful information.
The closest adversarial discourse work constrains the design. Huang and Li adapt from explicit to implicit relations within PDTB newswire. They use separate source and target encoders plus reconstruction, reaching 40.90% macro F1, 9.53 points over an explicit-only model and 5.7 over the standard adversarial method DANN. Their result shows that unconstrained alignment erases useful signal, but they never cross genres. Qin et al. train an implicit-relation model to imitate a connective-informed teacher within PDTB 2.0, again without genre transfer. Zhang et al. use adversarial learning to make predicted discourse trees look globally realistic, not to adapt across domains. They also report training instability, which we treat as a reason to keep the adversarial component small.
Two negative results restrict our options. Yung et al. found that synthetic target data generated by Mistral-7B, Llama 3.1 and Gemma 2 did not beat a news-only model on DiscoGeM. LLMs produce overly textbook examples, and news-trained filters reject genuine target patterns. ATOP shows that adversarial prompt-tuning can lose to simpler feature-based systems on structurally rigid essays. This warns us that alignment alone may not suffice.
Research Program
The backbone is a DeBERTa encoder over argument pairs. We choose it because Kobayashi et al. found span-masking pre-trained models to be the strongest encoders for discourse structure. A classifier predicts level-2 PDTB senses. Source data is labeled PDTB 3.0 implicit relations. Target data is unlabeled raw text from each target genre: the GUM documents behind GDTB, and the Europarl, novel and Wikipedia text behind DiscoGeM.
The method has three stages.
The first stage estimates target relation frequencies. We run a connective identifier and sense classifier, trained on PDTB explicit relations, over the raw target text. This yields an estimated explicit-sense distribution $\hat p^{\mathrm{exp}}_t(c)$ for each sense $c$. We then convert it to an implicit estimate:
$$ \hat p^{\mathrm{imp}}_t(c) \propto \frac{p^{\mathrm{imp}}_s(c)}{p^{\mathrm{exp}}_s(c)}\,\hat p^{\mathrm{exp}}_t(c). $$
Here $p^{\mathrm{imp}}_s$ and $p^{\mathrm{exp}}_s$ are the implicit and explicit sense frequencies in PDTB. The key assumption is that the ratio between how often a relation is left unmarked and how often it is marked is roughly stable across genres. We compare this estimator with the classical alternative, which needs no connectives: iteratively re-estimating priors from the source classifier's predictions on target text, an expectation-maximization procedure. We also test a combination that uses the connective estimate to initialize that procedure.
The second stage is prior-corrected conditional alignment. Let $f$ be the encoder features, $g$ the predicted sense distribution, and $D$ the genre-guessing network applied to $f \otimes g$ as in CDAN. Each source example with gold sense $y$ receives weight $w_y = \hat p^{\mathrm{imp}}_t(y)/p^{\mathrm{imp}}_s(y)$. The adversarial loss becomes
$$ \mathcal{L}_{\mathrm{adv}} = -\,\mathbb{E}_{(x,y)\sim s}\big[w_y \log D(f \otimes g)\big] - \mathbb{E}_{x\sim t}\big[\log(1 - D(f \otimes g))\big]. $$
The reweighting makes the source side look as if it had the target genre's relation mix. The encoder is therefore no longer pushed to make, say, temporal-heavy talk transcripts look elaboration-heavy. The classification loss is reweighted with the same $w_y$, and a reconstruction term in the style of the Genre Separation Network is added. The weights are re-estimated once after a first round of adaptation.
The third stage handles ambiguity. Because DiscoGeM items often carry several valid senses, target conditioning uses the full predicted distribution $g$ rather than its most likely sense. We evaluate against DiscoGeM's distributional labels as well as single labels. We also test CDAN's confidence weighting, which may hurt here because it systematically down-weights genuinely ambiguous relations.
Baselines
The naive reference is a DeBERTa classifier fine-tuned on PDTB implicit relations and applied unchanged to each target genre. GDTB reports 44.57% accuracy for a PDTB-only model on its genres. That number came from the GDTB authors' setup, so we retrain our own source-only model on the identical split and backbone to set the floor.
The strongest unsupervised competitors are DANN and CDAN+E, run on the same backbone and the same unlabeled target text. Each gets the same number of training steps and the same budget of 20 hyperparameter trials, selected on a held-out source-only validation set so that no method sees target labels. CDAN+E reports 87.7% on the Office-31 image benchmark and 65.8% on Office-Home. We found no reported result on Lacuna for CDAN on discourse relations. The closest discourse number is Huang and Li's 40.90% macro F1, from a different within-PDTB setting, which we cite only for context. Beating CDAN+E means a higher target macro F1, averaged over GDTB genres and the three DiscoGeM genres across five seeds, with the gain concentrated in genres whose relation frequencies diverge most from news.
Domain-adaptive masked-language-model pretraining on target text is run with the same number of target tokens and is also combined with each adversarial method. A "pseudo-implicit" baseline trains on target-genre explicit relations with the connective deleted. It uses exactly the connective signal our estimator uses, which isolates whether connectives help more as a frequency estimate than as direct training data. Supervised joint PDTB+GDTB training (64.40% reported by GDTB) serves as an upper bound, not as a competitor.
The ablations isolate the frequency correction. Setting all weights to one recovers plain CDAN. Weights computed from gold target frequencies give an oracle. Weights from the connective-free estimation procedure test whether connectives are needed at all. Removing the reconstruction term tests the guard against information loss.
Why not just…?
Why not just use a bigger pretrained model? In Evaluating Document Coherence Modelling, ALBERT-xxLarge detects out-of-place sentences with 96.9% accuracy on CNN news. Trained on CNN and tested on Wikipedia, it drops to 66.6%, against 81.7% for a model trained on Wikipedia itself. That pattern suggests scale buys in-genre templates, not genre-independent coherence. Even so, we rerun our best configuration with DeBERTa-large. If the larger source-only model closes most of the gap, the frequency correction is not earning its complexity.
Why not just continue masked-language-model pretraining on the target genre? Using Discourse Connectives to Test Genre Bias in Masked LMs found that this kind of continued pretraining on editorials shifted BERT's connective expectations toward the genre's bias and had no significant effect for RoBERTa. More basically, it adapts vocabulary but does nothing about relation frequencies. It is cheap and might stack with our method, so we report it alone and combined.
Why not just train on target explicit relations with connectives removed? This is the most serious threat, because it uses the same target signal more directly. The risk on the other side is that explicit-derived examples differ systematically from genuine implicit ones. Huang and Li's need for adaptation even within newswire points that way. If this baseline matches our method, we will conclude that connectives are best used as data, and we will merge the two by adding stripped explicit examples to the source side.
Why not just generate synthetic target data with an LLM? Yung et al. tested this on DiscoGeM with three open LLMs and found no consistent gain over news-only training. We cite their result rather than rerun it at full scale, but we reproduce one configuration on our split for comparability.
Proposed Experiments
The main evaluation trains on PDTB 3.0 and adapts separately to each GDTB genre and to the Europarl, novel and Wikipedia sections of DiscoGeM 1.5. We report accuracy (to match GDTB), macro F1, per-sense F1 for the temporal and contingency senses, and, on DiscoGeM, cross-entropy against the distributional labels. Spoken GDTB genres are analysed separately, because connective detection on disfluent speech is likely to be noisier.
The first decisive check is a controlled frequency-shift experiment. We resample the target test sets to vary the divergence between source and target relation frequencies while holding wording fixed, then plot target macro F1 against that divergence for DANN, CDAN+E and our method. The core claim predicts that the unweighted methods fall as divergence grows while ours stays flat. If the curves are parallel, frequency shift is not the mechanism and the claim is wrong.
The second check validates the estimator. On GDTB, where gold explicit and implicit labels exist per genre, we measure the distance between estimated and true implicit frequencies for the connective-based estimator, the connective-free estimator and their combination. We then relate estimation error to the accuracy gap between estimated and oracle weights.
The third check is the full ablation table under matched compute. It would establish whether the gain comes from the frequency correction, from class conditioning, or from reconstruction.
References
- GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domains
- DiscoGeM: A Crowdsourced Corpus of Genre-Mixed Implicit Discourse Relations
- Discourse Sense Flows: Modelling the Rhetorical Style of Documents across Various Domains
- Conditional Adversarial Domain Adaptation
- Genre Separation Network with Adversarial Training for Cross-genre Relation Extraction
- Unsupervised Adversarial Domain Adaptation for Implicit Discourse Relation Classification
- Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification
- Adversarial Learning for Discourse Rhetorical Structure Parsing
- Synthetic Data Augmentation for Cross-domain Implicit Discourse Relation Recognition
- Adversarial Topic-Aware Prompt-Tuning for Cross-Topic Automated Essay Scoring
- A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing
- Evaluating Document Coherence Modelling
- Using Discourse Connectives to Test Genre Bias in Masked Language Models