Copy the Word, Question Its Sentiment: A Polarity Gate for Cross-Domain Triplet Extraction

When a review model trained on restaurants reads laptop reviews, it usually finds the right words and then reads them the wrong way. It can locate "unpredictable" and "the handling" without trouble. It then assigns "unpredictable" the positive reading it picked up from movie-like contexts, even though unpredictable handling is bad. We propose a decoder that separates these two steps. The first step is a pointer-generator, a text generator that at each step either writes a word from its vocabulary or copies a word directly from the input. It copies the aspect ("handling") and the opinion ("unpredictable") verbatim. When it then has to output a sentiment label, a second learned gate chooses between two sources. One is the polarity the copied opinion words usually carry, read from those words alone. The other is a polarity recomputed from the aspect and the surrounding sentence. That second gate is trained, without target-domain labels, to close on words whose polarity flips across domains.

The payoff is a mechanism aimed at the error that dominates domain transfer, rather than one more global alignment loss. Prior work suggests the extraction half transfers and the interpretation half does not. Pointer-generators carry over what to copy between news and opinion text but not how to phrase subjective content (A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization). Generative aspect-sentiment models handle span elements well and polarity poorly (LEGO-ABSA). Our proposal makes the "inherit or recompute" decision explicit, adapts only the part that needs adapting, and lets us measure where transfer breaks.

Core Claim

The task is aspect sentiment triplet extraction: pulling (aspect, opinion, polarity) tuples out of a sentence. In cross-domain transfer, most triplet errors come from polarity on a small set of opinion words whose meaning depends on the domain, not from span boundaries. A decoder that copies spans and then gates between inherited and recomputed polarity should improve cross-domain triplet F1 over the strongest extractive adaptation method. Nearly all of that gain should fall on the domain-dependent subset, and span F1 should stay unchanged. If a plain copy-augmented generator with the same unlabeled-target training matches it, the gate is unnecessary. We treat that outcome as the main threat, not a footnote.

What Exists and What Differs

Two systems come close. PASTE decodes triplets with pointer networks that select span start and end positions, and it sends pooled span vectors to a separate feed-forward classifier for polarity. The pointers never mix with a generation vocabulary, and the polarity decision cannot use the fact that a word was copied. PASTE is also evaluated only in-domain. LEGO-ABSA linearizes all aspect-sentiment tasks as T5 generation with slot prompts. It has no copy gate, and its authors name polarity as the weakest element. Our design differs from both in one specific way. Polarity is itself a two-source mixture whose "copy" branch is the prior polarity of the copied tokens, and the switch between branches is trained on evidence of domain dependence.

Three lines of work supply the ingredients. The domain-dependence switch comes from Transfer Learning for Multiple-Domain Sentiment Analysis. That model uses a latent binary variable per word to mark whether its polarity is fixed or domain-specific, and it shows that "unpredictable" and "sharp" flip. We turn that bag-of-words latent into a supervision target for a neural gate. The target-domain training signal comes from the pivot "hiding trick" of Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification and Leveraging Auxiliary Tasks for Document-Level Cross-Domain Sentiment Classification. These papers mask universal words like "excellent" and "terrible" and predict them from context, which forces domain-specific phrases to take on their polarity. The idea of manipulating the copy gate directly comes from Dr. Summarize, which penalizes generation to force faithful copying and adds a separate switch for negation. We borrow its penalty for span slots and its separate-switch design for polarity.

Two papers constrain the design. FOAL shows that domain-adversarial alignment blurs category boundaries, and it reports that its own contrastive method depends on the quality of pseudo-labels. We therefore avoid adversarial alignment and avoid self-training on the source model's polarity guesses. Polarity inversion operators in PLM finds that negation in BERT embeddings is steerable but not cleanly invertible. So the inherited-polarity branch must read the whole copied opinion span, including any negator, rather than a head word with a flip applied. We also report negated triplets separately.

Mechanism

The backbone is a pretrained BART-style encoder-decoder that outputs linearized triplets. It carries a standard pointer-generator head, which A Copy-Augmented Generative Model for Open-Domain Question Answering shows adds only a small number of parameters. In aspect and opinion slots we apply a generation penalty in the style of Dr. Summarize, so those tokens are copied. In the polarity slot, the distribution over {positive, negative, neutral} is

$$P(y) = g\,P_{lex}(y \mid o) + (1-g)\,P_{ctx}(y \mid a, o, x)$$

The terms are defined as follows. $o$ is the copied opinion span. $P_{lex}$ is a small head that sees only the copy-attention-weighted embeddings of $o$ and has no sentence context, so it estimates "what this phrase usually means." $a$ is the copied aspect and $x$ the full sentence. $P_{ctx}$ is the decoder's contextual prediction. $g \in [0,1]$ is a gate computed from the decoder state and a learned domain embedding.

The gate gets weak supervision toward $\hat z$, an estimate of whether the opinion word's polarity differs between source and target. We compute $\hat z$ by comparing each word's polarity in the labeled source data with its polarity in unlabeled target text. The target polarity is inferred from how often the word co-occurs with positive and negative pivot words, using the weighted log-likelihood ratio scoring of the pivot papers. The Bayesian switch model is an alternative estimator, and we compare the two. On target text we also train $P_{ctx}$ with the pivot-hiding objective. The contextual branch therefore learns the target domain's polarity conventions exactly on the words the gate will send to it. Source-domain triplet supervision trains everything end to end.

Research Program

The work has three parts. First, we measure the problem. We train strong in-domain models, run them across the transfer pairs, and split errors into span errors and polarity-given-correct-span errors. Within polarity errors, we split again into opinion words with high versus low $\hat z$. If polarity errors are not concentrated on high-$\hat z$ words, the premise fails and we report that.

Second, we build the gated decoder and its ablations. Third, we analyze the gate. The checks are whether $g$ tracks $\hat z$ on held-out words, whether it closes on negated spans, and whether the gate predictions line up with the known flipping words from the multi-domain literature.

Baselines

The naive reference point is a source-only copy-augmented generator: the same BART backbone and pointer head, with the polarity slot generated ordinarily, trained on source only. This measures how much transfer we get for free.

The strongest existing method is FOAL, which reports an average triplet F1 of 50.12 across six restaurant/laptop transfer scenarios, against 47.09 for its Span-ASTE backbone (FOAL). It is the strongest competitor because it is built for exactly this cross-domain setting and explicitly targets aspect-opinion-polarity pairing. We will rerun FOAL on the same six transfer splits and the same unlabeled target pool, with the same tuning budget in trials and GPU hours, and report our reproduction next to the published 50.12. Beating it means a higher average triplet F1 than our reproduction across three seeds, with the difference exceeding seed variance. It should also include a clear margin on the high-$\hat z$ subset.

We will also run LEGO-ABSA-style T5 prompting and PASTE as the generative and pointer-based references. The file gives only relative gains for these, with no absolute cross-domain number; no reported result was found on Lacuna for either method in cross-domain triplet extraction.

The key ablations isolate the gate. The first fixes $g=0$ but keeps pivot-hiding training on target, which amounts to a copy generator with the same unlabeled-target signal. The second fixes $g=1$, which uses the lexical prior only. The third learns $g$ without $\hat z$ supervision. The fourth removes the pivot-hiding loss.

Why not just…?

Why not just fine-tune T5 or BART to generate triplets, as LEGO-ABSA does? Pretrained generators already copy spans well, so a copy gate on spans alone would add little. LEGO-ABSA itself locates the remaining weakness in polarity, not spans. Our contribution sits at the polarity slot, and the $g=0$ ablation is exactly "fine-tuned generator plus our target signal." That ablation is a real threat. If it matches the full model, we conclude that pivot-hiding does the work and the gate is only a diagnostic tool, and we will say so.

Why not just use FOAL's contrastive alignment, or add it to a generator? FOAL depends on target pseudo-labels from the source model, and those are least reliable on polarity-inverted words, the very cases that matter here. Our target signal comes from pivot co-occurrence rather than from the model's own guesses. A variant of our model with FOAL's pair-level contrastive loss added tests whether the two signals are complementary or redundant.

Why not just do domain-adaptive pretraining on target reviews? When FLUE Meets FLANG shows large gains from in-domain pretraining for sentence-level financial sentiment: 91.9% versus 87.2% accuracy on Financial PhraseBank. Continued masked-language pretraining on the target review pool is therefore a fair competitor, and we will run it under matched compute. Our expectation is that it helps spans and general fluency but does not specifically fix aspect-conditioned flips. The high-$\hat z$ subset analysis will show whether that expectation holds.

Why not just attach a classifier head on span representations, as PASTE does? A classifier on pooled spans cannot be told when to distrust the word's usual meaning. Its output is one entangled judgment. Replacing our polarity mixture with such a head, trained with the same target losses, tests whether the explicit decomposition matters beyond parameter count.

Proposed Experiments

The main benchmark is ASTE-Data-V2 in its six cross-domain restaurant/laptop scenarios, following FOAL's protocol. We report triplet F1, span F1, polarity accuracy given correct spans, and all three on the high-$\hat z$, low-$\hat z$ and negated subsets.

For pivot statistics and for validating $\hat z$, we use the multi-domain Amazon sentiment dataset of Blitzer et al. We check that $\hat z$ recovers flips such as "unpredictable" across the Books, DVD, Electronics and Kitchen domains before trusting it as gate supervision. As a stress test with little overlap in vocabulary, we transfer restaurant/laptop models to the RES/LAP/DEV opinion-extraction data of TIMN, scored on spans only because that data lacks polarity.

The decisive check has three parts. First, the full model must beat both the reproduced FOAL and the $g=0$ ablation on average triplet F1. Second, that margin must come mostly from polarity on the high-$\hat z$ subset while span F1 stays flat. Third, the learned gate must correlate with $\hat z$ on opinion words never seen in training. If the first part holds without the second and third, the gain is not coming from the mechanism we claim.