Global Relations, Local Polarity: Federated Sentiment Triplet Extraction

Consider a model that reads a review and pulls out every (aspect, opinion, polarity) triple, for example ("food", "spectacular", positive) and ("service", "nightmare", negative). Such a model has to learn two quite different kinds of knowledge. The first is how the triples in one sentence constrain each other: a "but" usually flips polarity between two aspects, and neighbouring aspects tend to share sentiment. The second is what individual opinion words mean in a given place: "hot" is praise for soup and a complaint about a laptop. Suppose these reviews sit with separate data owners who cannot pool them. That is federated learning, where each owner trains on its own data and only model updates are averaged on a server. We argue the two kinds of knowledge should be treated differently. The relational knowledge is shared across owners and benefits from averaging. The word-to-polarity knowledge varies by owner and is damaged by averaging.

The proposal builds a federated triplet extractor with a clean split. The layer that lets candidate triples interact is trained globally. A small opinion-to-polarity correction is kept on each client, never transmitted, and re-estimated from a handful of local examples. We test the split directly with two diagnostic test sets: one for domain-dependent polarity shifts and one for contrasting multi-aspect sentences. The central prediction is that each half of the model fixes a different one of these failures.

Core Claim

In federated multi-domain sentiment extraction, the differences between clients are concentrated in the mapping from opinion spans to polarity. The rules of interaction between triples inside a sentence transfer across clients. It follows that global training of the relational layer, combined with lightweight local reconstruction of a polarity correction, should do three things. It should match the per-domain accuracy of full local personalization. It should keep the contrast-resolution ability that full local fine-tuning erodes on small clients. And it should handle a new client from a few labeled sentences. If full fine-tuning after averaging, or a simple domain indicator, matches it on both diagnostic sets, the claim is false.

Where This Sits

Only one paper in the file addresses federated aspect sentiment directly. Improving Federated Learning for Aspect-based Sentiment Analysis via Topic Memories (TM-FL) identifies domain polarity shift, the "hot" example, as the main failure of plain federated averaging. It has three limits that matter here:

  • Domain conditioning is global. It conditions a server-side classifier on a topic vector from a topic model pretrained on public reviews.
  • Aspects are classified one at a time. There is no opinion extraction and no interaction between aspects.
  • Hidden states leave the client. It uses split learning: encrypted hidden states are sent to the server on every forward pass.

TM-FL motivates the proposal, since it gives evidence that polarity shift is real under federation. We depart from it in three ways. We handle shift with client-local parameters rather than a global topic prior. We extract full triples. And we transmit only parameter updates.

The relational side supplies the method.

  • ASTE-Transformer builds candidate aspect–opinion pairs on a DeBERTa backbone. It then lets candidate triples attend to one another in a transformer layer. Removing that layer costs up to 5 F1 points. That layer is exactly what we propose to share globally.
  • Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation shows that neighbouring aspects cluster in sentiment. It raises DeBERTa from 67.71% to 74.47% accuracy on the adversarial Laptop14-ARTS set.
  • RR-GCN models aspect-to-aspect relations explicitly and reports 82.1 F1 on REST14.

These are further evidence that intra-sentence structure carries signal a topic vector cannot. All three are strictly centralized.

Federated Reconstruction supplies the federated mechanism. It splits parameters into global and never-transmitted local parts. The local part is rebuilt by a few gradient steps on a client support set with the global parameters frozen. It was shown on matrix factorization, where it reports 0.907 RMSE on MovieLens 1M, and on next-word prediction.

It also constrains the design. It warns that reconstruction becomes noisy with tiny support sets. That is why our local part is a low-rank correction and not a whole classification head.

The two "relational federated learning" papers in the file are not competitors. TablePuppet handles database tables, and basis alignment handles knowledge-graph embeddings.

Research Program

Base model. We use an ASTE-Transformer-style extractor: an encoder, a pair-construction layer, and the triplet interaction layer. We deliberately leave out parser-dependent graph modules, because parsing is fragile and costly at the edge.

Scoring rule. For a candidate triple $t$ with interaction-layer output $r_t$ and opinion-span embedding $e_o$, client $c$ scores polarity as

$$p_c(y \mid t) = \mathrm{softmax}\left(W r_t + A_c B_c e_o\right),$$

where the symbols are:

  • $W$ is the global polarity projection.
  • $A_c B_c$ is a rank-$k$ matrix local to client $c$, with $k$ around 4.

What stays where.

  • The encoder, pair layer, interaction layer and $W$ are aggregated with FedAvg or an adaptive federated optimizer.
  • $A_c, B_c$ start at zero and are reconstructed statelessly each round. They get a few steps on a support split of the client's data. The global update is then computed on the query split, following Federated Reconstruction.
  • Raw text and hidden states never leave the client.

Settings. We treat this as cross-silo federation: a handful of review platforms or verticals, each able to run a DeBERTa-base forward pass. A smaller-backbone variant reports the communication cost.

Diagnostic test sets. Two are mined automatically from gold labels.

  • Shift set. Test triples whose opinion word carries opposite polarities in different training domains.
  • Contrast set. Test sentences with two or more gold triples of differing polarity, supplemented by MAMS for aspect-level polarity.

The claim predicts a specific pattern. The local correction should lift the shift set. The global interaction layer should protect the contrast set. And moving the interaction layer to the client should hurt the contrast set on small clients.

Baselines

Naive reference points. Local-only training trains the same extractor on each client's data alone. It is the "do nothing collaborative" point, and it shows how badly small clients suffer. The centralized upper bound pools all data and trains ASTE-Transformer with the same backbone and budget.

Strongest expected competitor. This is FedAvg of the full ASTE-Transformer, followed by local fine-tuning of all parameters on each client. It is the simplest personalization recipe and has no structural assumption.

Domain-conditioning baselines. First, FedAvg with a learned domain-indicator embedding added to the polarity logits. This is a cheaper, oracle-informed stand-in for TM-FL's topic vector. Second, TM-FL itself on the aspect-classification subtask, where the file gives reported numbers: TM-FL reaches 82.29% accuracy on LAP14 against 81.35% for standard federated learning (TM-FL).

Reported numbers for the triplet task. No reported result was found on Lacuna for federated triplet extraction on any benchmark. The file also gives no itemized triplet F1 for ASTE-Transformer. The only centralized triplet numbers available are 43.50–53.62% F1 across the four SemEval sets from Knowing What, How and Why. We therefore rerun all triplet baselines ourselves under identical conditions:

  • the same client partitions;
  • the same backbone;
  • the same number of communication rounds;
  • an equal budget of local gradient steps for any personalization;
  • the same hyperparameter search budget, tuned on a held-out validation client.

Ablations isolating the split. The first removes the interaction layer, so triples are scored independently. The second makes the whole polarity head local rather than a low-rank correction. The third reverses the split, keeping the interaction layer local and polarity global.

What counts as beating the baselines. Our method must satisfy three conditions:

  • mean triplet F1 at least matching fine-tuned FedAvg;
  • a statistically significant gain on the contrast set for the smallest clients;
  • higher F1 than the domain indicator on an unseen-domain client.

Why not just…?

Why not just run FedAvg on the relational model and fine-tune locally? This is the real threat. With enough local data, full fine-tuning may capture polarity shift without harming anything. Our bet is that on small clients it overwrites the interaction layer, which was learned from everyone's contrastive sentences.

The experiment settles it. We sweep client size by subsampling each client from 50 to the full set of sentences, with equal local step budgets, and track the contrast set separately. If fine-tuning matches us on the contrast set at every size, we will report that the partition is unnecessary.

Why not just condition on domain, as TM-FL does with topic memories? A domain signal resolves "hot" only at domain granularity. It needs either an oracle label or a centrally pretrained topic model. It cannot help a client from a domain unseen in training. It also does nothing for same-domain contrast, where both aspects share a topic, which is exactly the case that inter-triple interaction targets (ASTE-Transformer).

The honest concern is that the SemEval heterogeneity may be purely domain-level. If so, the domain indicator could tie with us on seen clients. The leave-one-domain-out test with a small support set is where the two should separate.

Why not just use an independent-aspect model under FedAvg? Standard federated learning on single aspects is the setting TM-FL already studied. Its 81.35% on LAP14 says nothing about linking opinions to aspects.

Evidence from centralized work suggests that removing relational structure costs real accuracy:

Our no-interaction ablation measures whether that loss persists, or grows, under federation.

Why not just fine-tune a generative model to emit triples? UGTS reports substantial gains of structured tagging over generative methods for quad prediction. Generation also inflates per-round communication. We therefore do not pursue it, beyond noting it as a possible later comparison.

Proposed Experiments

Data and clients. We use the four SemEval triplet sets from Knowing What, How and Why. There are three partitions:

  • Domain partition: laptop versus restaurant clients.
  • Fragmented partition: each domain split into many small non-overlapping clients, to stress reconstruction under sparse local data.
  • Leave-one-domain-out: laptops held out as a new client with only 20–100 labeled sentences.

MAMS supplies an extra contrast probe at the aspect level. Laptop14-ARTS supplies an adversarial robustness check, with the centralized reference of 74.47% for LSA-enhanced DeBERTa from LSA.

Metrics. We report five:

  • exact-match triplet F1, as a mean and for the worst client;
  • F1 on the shift and contrast sets;
  • F1 on the new client as a function of support-set size;
  • bytes communicated per round;
  • local compute per reconstruction.

Decisive checks. Three results would support the claim together:

  1. Our method beats fine-tuned FedAvg on the contrast set for small clients while tying on overall F1.
  2. The reversed-split ablation loses on the contrast set, while the no-local-correction ablation loses on the shift set.
  3. On the held-out domain, the local correction beats the domain indicator.

If none of the three holds, the partition hypothesis is wrong, and the lesson is that federated triplet extraction needs no special structure beyond fine-tuning.