Refining Only What Disagrees: Topic-Sentiment Consistency as a Cheap Stopping Signal for Quadruple Extraction

Fine-grained opinion mining asks a model to read a review sentence and return every opinion in it as a structured record: which aspect is discussed, which category it belongs to, which words carry the opinion, and whether that opinion is positive, negative or neutral. The best current systems generate these records in one left-to-right pass, and they are still wrong roughly two times in five. Asking the model to critique and rewrite its own answer is the obvious remedy, but each round of rewriting multiplies inference cost. Each round also risks turning a correct answer into a wrong one. We propose a different trade. After one fast draft, a small checker asks a narrow question about each extracted record: do its parts fit together? Would this opinion word, about this category, plausibly carry this polarity? Only the parts that fail this check are regenerated, with everything else held fixed. Refinement then costs a few decoded tokens instead of whole extra passes, and the check doubles as the stopping rule.

The idea draws on the oldest finding in joint sentiment-topic modeling: what is discussed and how it is judged are statistically entangled. "Cold" is good for beer and bad for a hotel room, so an extracted record whose polarity contradicts its category and opinion word is evidence of an error. We turn that dependence, usually treated as something the model must learn, into a check that runs at test time. This gives refinement loops on a subjective task the thing they usually lack, which is a way to tell an already-correct answer from a wrong one without a ground-truth verifier.

Core Claim

On aspect-category-opinion-sentiment extraction (ACOS: the four-part records described above), a draft-then-repair system can recover most of the accuracy gain of always-on iterative refinement at well under the cost of a second full pass. Three conditions must hold. The decision of which records to revisit must come from a compatibility score over the joint (category, opinion, sentiment) structure, not from raw decoder confidence. Repair must re-decode a single slot rather than the whole output. And the loop must stop as soon as every record passes. We also claim that this gate protects neutral and implicit records, the classes known to degrade, from being "corrected" into polarized ones.

Positioning Against Prior Work

The generators we build on are single-pass. Generative Sentiment Analysis via Latent Category Distribution and Constrained Decoding trains a T5 model that writes quadruples as text. It adds a latent category distribution to the encoder and a prefix tree that forbids ill-formed outputs, reaching F1 0.613 on Restaurant-ACOS and 0.455 on Laptop-ACOS. We reuse its formulation and constrained decoding as our draft model; it never revisits a draft, and we add that. iACOS shows that adaptively chosen negative examples help implicit extraction. We borrow the idea of informative corruptions to train our checker rather than our extractor.

Two lines of work already use iteration inside opinion extraction, and both pay for it on every input. IMKTN passes messages between aspect, opinion and sentiment representations for several rounds, using capsule-style routing. MARN repeatedly refines a dialogue graph to link opinion elements scattered across turns. Both motivate our premise that the subtasks inform each other. Neither measures latency, and neither can skip refinement when the first answer is already right.

The refinement-latency literature supplies both the method and the constraints. Meta-Reflection folds reflection into one forward pass through a learned codebook inside the network, reporting first-token latency of 246ms versus 951ms for a retrieval-based reflection method on e-commerce intent detection. It needs white-box access and offers no control over output structure, and we pursue the opposite design: explicit, selective, schema-respecting repair. Critique-RL shows that a critic rewarded only for downstream improvement loses discriminability, meaning it can no longer recognize answers that are already correct, and over-edits as a result. Its fix depends on ground-truth verifiers, which subjective tasks lack. Our compatibility checker is an attempt to supply that discriminability signal for sentiment from the structure of the task itself. ICE iteratively edits Yelp reviews toward extreme sentiment. Its authors report two problems: high latency from repeated full sequence-to-sequence passes, and topical drift caused by the lack of a stopping signal. Both are failure modes our slot-level repair and consistency stopping rule are designed to avoid.

Research Program

The draft model is a T5 ACOS generator trained as in the LCD-CD setup, with prefix-tree constrained decoding. It emits quadruples greedily.

The checker scores each drafted quadruple $q=(a,c,o,s)$ in sentence $x$ with

$$g(q)=\log p_\phi(s \mid c,o,x)+\log p_\phi(c \mid a,x)+\lambda \log \pi(s \mid c,\tilde o).$$

The checker has two learned parts and one corpus-level part. The model $p_\phi$ is a small cross-encoder trained on gold quadruples and their corruptions: polarity flips, swaps to sibling or parent categories (for example restaurant#general, or "Hardware" for "Graphics"), and implicit/explicit swaps. The term $\pi(s \mid c,\tilde o)$ is a topic-conditioned polarity prior over category and opinion-word lemma $\tilde o$. It is estimated by a weakly supervised joint sentiment-topic model fit on unlabeled in-domain reviews, Yelp-style restaurant text and Amazon laptop text. The weight $\lambda$ is tuned on the development set. A quadruple is flagged when $g(q)$ falls below a threshold $\tau$, and the checker also names which term is lowest, which tells the repair step which slot to blame.

Repair re-decodes only the blamed slot. The other three elements are forced as a fixed prefix, the encoder states are cached, and the prefix tree constrains the slot to be well-formed. The re-decoded quadruple is re-scored. The loop stops when no quadruple is flagged or after a budget of $R$ rounds. Sweeping $\tau$ traces a full accuracy-latency curve, from zero refinement at one extreme to refining everything at the other.

The analysis has three parts. We count how often repair turns a correct quadruple wrong, broken down by polarity class, which tests the over-correction worry raised by Critique-RL. We report gate precision and recall against actual errors. We measure added cost in decoded tokens, forward passes and wall-clock time.

Baselines

The naive reference point is the draft model alone: a single constrained pass with no refinement, our reimplementation of the LCD-CD recipe. For the published version of this model, the paper reports F1 0.613 on Restaurant-ACOS and 0.455 on Laptop-ACOS (LCD-CD). It reports GAS at 0.596 and 0.422, and Extract-Classify-ACOS reaches 0.4461 on Laptop-ACOS. Our reimplementation must land near these numbers before any refinement result counts. iACOS reports only relative gains over BART-CRN (12.78% and 6.47% averaged F1), so we cite it as a strong extractive comparison and rerun it ourselves rather than quote an absolute figure. We found no reported latency result on Lacuna for any ACOS system, so every latency number in this study will be measured by us on one GPU, at batch size 1 and at a fixed larger batch size.

The strongest competitor on accuracy is always-on full refinement. The same T5 is fine-tuned to read its draft and rewrite the entire quadruple list, in the spirit of SRT-style self-refinement, for a fixed one to three rounds. This baseline answers whether refinement helps at all on ACOS and sets the accuracy ceiling we aim to approach.

The strongest competitor on efficiency is distillation. A single-pass student is trained on the outputs of full refinement, which is the explicit, black-box analogue of Meta-Reflection's amortization. All systems share the same backbone size, the same training data, the same unlabeled corpus wherever one is used, and the same hyperparameter search budget of a fixed number of development-set trials.

The ablation that isolates our key ingredient replaces $g(q)$ with the draft decoder's own mean token log-probability for each quadruple, keeping slot-level repair and the stopping rule unchanged. A second ablation sets $\lambda=0$ to test whether the corpus-level topic-sentiment prior matters. A third keeps our gate but regenerates the whole output instead of one slot.

Beating these baselines means meeting three conditions. Our system must exceed the single-pass draft by a statistically significant margin across five seeds. It must recover at least two-thirds of full refinement's F1 gain at no more than 1.3 times single-pass wall-clock time. And it must lie strictly above the confidence-gated and distilled systems on the accuracy-latency curve.

Why not just…?

Why not just distill the refined outputs into a single-pass model? This is the real threat. If a student trained on refined outputs matches full refinement at single-pass cost, test-time repair is unnecessary. Meta-Reflection's results suggest that amortization can work on intent detection and code. Whether it works for exact-match structured outputs, where one wrong element sinks a whole record, is untested. We run the distilled baseline at matched compute. If it matches our F1, we will report that selective repair is unnecessary on ACOS. In that case the checker's remaining value is as a stopping and audit signal, and we will measure that separately.

Why not just gate on decoder confidence? Token probabilities are free, and they are the obvious gate. They measure how fluent the output is, not whether its parts agree. A confidently decoded "positive" attached to a hedged, neutral opinion is exactly the kind of error that ECAN's authors (ECAN) describe on the neutral class. The confidence-gated ablation settles this directly. If it matches our gate in precision and recall against actual errors, our central claim fails.

Why not just refine everything for a fixed number of rounds? Fixed rounds spend compute on the majority of sentences whose drafts are already correct. ICE (ICE) and Critique-RL (Critique-RL) both document that unchecked loops drift and over-correct. Our correct-to-wrong flip-rate measurement, broken down by polarity, will show whether always-on refinement damages neutral and implicit records, and whether gating prevents that damage.

Why not just use a larger single-pass model or wider beam search? We include beam sizes 1, 4 and 8 for the draft model and a model one size larger as points on the latency curve. If either dominates our curve, the selective-repair premise is weakened, and we will report it.

Proposed Experiments

The primary benchmarks are Restaurant-ACOS and Laptop-ACOS, scored by strict exact-match quadruple F1. We report breakdowns for implicit aspects, implicit opinions and neutral polarity. Latency is reported as wall-clock time per sentence, decoded tokens and forward passes. The secondary test is DiaASQ English, where MARN reports Micro-F1 of 44.95% (MARN). Opinion elements there are scattered across turns, so inconsistent records should be more common, and the test asks whether the same checker transfers without redesign.

Three checks are decisive. First, at matched cost, the compatibility gate must flag actual errors with higher precision than confidence gating. Second, accuracy against latency, swept over $\tau$, must dominate the distilled baseline and fixed-round refinement in the range from 1.0 to 1.5 times single-pass cost. Third, on correct drafts, the flip rate of neutral quadruples under our system must be lower than under always-on refinement.

If gate precision does not beat confidence gating, or distillation matches our curve, the proposal's central mechanism is refuted. We would then report that topic-sentiment consistency does not carry useful error information beyond what the decoder already knows.