Aspect sentiment triplet extraction (ASTE) takes a review sentence and lists every (aspect, opinion, sentiment) triple it contains. "The battery lasts long but boot-up takes long" yields (battery, lasts long, positive) and (boot-up, takes long, negative). Small specialized taggers do this well on short benchmark sentences. They break down on long real-world reviews, on new product domains, and in cases where the same opinion word flips polarity depending on its target. Large language models (LLMs) should be strong in exactly those cases. Yet when GPT-4 is prompted to reason step by step and write out the triplets, it loses to a 0.12B-parameter tagger. The reason is that it paraphrases the spans it is supposed to copy (MiniConGTS).
We propose to stop asking the LLM to write spans at all. A small tagger proposes candidate triplets, each tied to exact token positions. For each candidate, a retriever fetches labeled training triplets that look similar. It deliberately includes look-alikes whose polarity differs. The LLM then reasons step by step about whether each retrieved precedent transfers to the candidate. Its output is only a verdict over the candidates: keep, drop, or relabel the sentiment. It may add a missed pair, but only by pointing to token positions, so span drift cannot happen. The bet is that retrieval plus step-by-step reasoning helps ASTE only when the reasoning works on precedents, and precise spans come for free when the LLM never generates them.
What is known and where the gap is
The three ingredients have each been paired, but never in this way. Chain-of-thought (CoT) prompting means having a model write intermediate reasoning before its answer. It has helped aspect-level sentiment when the target is already given. THOR reasons in three hops (latent aspect, then implicit opinion, then polarity) for implicit sentiment. Syn-Chain walks GPT-4o through a dependency parse to classify polarity for given aspects. ZeroABSA adds a domain-identification step for cross-domain classification. None of the three must discover aspect and opinion boundaries, which is the hard part of ASTE. These papers motivate the use of reasoning for polarity decisions. The constraining result is the MiniConGTS finding that free-form CoT degrades exact-span extraction, and that result rules out any design in which the LLM emits span strings.
Retrieval-augmented generation (RAG) means fetching relevant stored text and placing it in the model's input. Retrieval has entered ABSA in three ways:
- RLI builds a store of training triplets and blends retrieved labels into a BERT classifier's representation. It reports that semantic retrieval falls into a "retrieval trap": neighbors that share wording but have opposite polarity.
- Wang et al. retrieve in-context examples by syntax and aspect-sentiment structure, not by topic. They do this for aspect extraction and classification only.
- IT-RER-ABSA co-trains a retriever with a Flan-T5 generator for aspect-sentiment pairs.
None of the three reasons about whether a retrieved neighbor applies. RLI averages over neighbors, and the other two copy them as demonstrations. We reuse RLI's triplet-level store and Wang et al.'s observation that structure matters more than topic. Where they blend or copy neighbors, we make the polarity-trap case an explicit object of reasoning.
The closest system is Li et al.'s black-box quad alignment. A fine-tuned small model generates about ten candidate quadruples, and the LLM selects among them under a learned reward. Our "tagger proposes, LLM decides" structure shares that skeleton. It differs in three ways. Our LLM reasons step by step over retrieved per-candidate precedents, while theirs selects directly. Our candidates are token-indexed, so the output is exact by construction. And our retrieval is built to surface contradicting precedents. Li et al.'s system is therefore the baseline we most need to beat, and the ablation "same pipeline, no reasoning" is designed to separate our contribution from theirs.
Mavi et al. supply the methodological lesson, from financial and tax question answering: retrieval that preserves structure prevents reasoning failures. Our analogue of preserving structure is retrieving triplets with their span positions and connecting context, not retrieving whole sentences.
Core Claim
On ASTE, an LLM adds value as a judge of candidate triplets, and only under two conditions. First, it must see per-candidate precedents chosen to include both agreeing and contradicting labels. Second, it must explain in writing why a precedent's polarity does or does not transfer. Under those conditions, we expect a tagger-plus-judge system to match the tagger on short in-domain benchmarks. We expect it to beat the tagger clearly on three slices: long multi-domain reviews, cross-domain transfer, and a "trap" subset where the nearest neighbor has the opposite polarity. Removing either the reasoning or the contrastive retrieval should erase most of that gain.
Research Program
Stage one: candidate proposal. We train a grid tagger, MiniConGTS or ASTE-Transformer (ASTE-Transformer). We lower its decision threshold so it emits a high-recall set of candidate triplets, each with token indices and a confidence. We first measure the recall ceiling: the fraction of gold triplets that appear among the candidates. That ceiling bounds everything downstream. Long reviews are split into overlapping windows so candidates can still pair aspects and opinions that sit far apart.
Stage two: contrastive precedent retrieval. For each candidate, we build a retrieval key by encoding three things together: the aspect span, the opinion span, and the words between them, where negations and contrast words such as "but" live. We query a store of labeled training triplets. The store is in-domain where labels exist; for cross-domain experiments it is a small labeled pool from the target domain. Retrieval does not return the plain top-k. It returns the top neighbors within each polarity class, so a candidate like (boot-up, takes long) arrives alongside both "long battery life (positive)" and "long wait (negative)." This turns the retrieval trap from a hidden hazard into evidence the judge can inspect.
Stage three: reasoned verdict. The LLM receives the indexed sentence, the candidates, and their precedents. It follows a fixed reasoning template:
- Restate each candidate by token index.
- Describe what links the aspect and the opinion, including any negation, contrast, or shared modifier.
- For each precedent, say whether its polarity transfers and why.
- Issue a verdict.
A constrained output schema accepts only candidate identifiers, polarity labels, and index ranges. The output spans are therefore copied from the input sentence and cannot drift. Shared-opinion and coordination cases ("the chicken and the beef were both overcooked") are judged jointly within one call, because these overlaps are where ASTE-Transformer found independent decisions fail.
Analysis. We will audit the written reasoning to see whether it is faithful. THOR documents that reasoning chains can be wrong even when the label is right. We test faithfulness by perturbing the precedents, for example by flipping the labels of retrieved neighbors, and checking whether verdicts and rationales change accordingly. A judge that ignores its precedents would expose the reasoning as decoration.
Baselines
The naive reference point is the stage-one tagger alone at its tuned threshold, trained on the same labeled data. If the judge cannot beat its own proposer, the pipeline has no value.
The strongest existing in-domain method we can anchor numerically is RLI. It reports 74.34 triplet F1 on the D_b Restaurant-14 split (RLI). It is the most relevant competitor because it also retrieves training triplets but blends their labels without reasoning. We will rerun it on every split we use, with its released configuration, and we will report both its published number and ours. MiniConGTS and ASTE-Transformer are run as second strong taggers. The file reports only relative gains for them, so we will report our reproduced absolute numbers. For DMASTE we found no reported per-model result on Lacuna, so all DMASTE baselines are our own runs.
The closest LLM competitor is a reimplementation of Li et al.'s candidate-selection approach for triplets. It gets the same candidate set, the same LLM, and direct selection without precedents or reasoning. Their reported 58.81% average quadruple F1 is on quadruples, not triplets, so it is not directly comparable. We will report it only as context.
We also run Wang et al.-style structure-aware few-shot prompting in which the LLM generates triplets freely. This is the "obvious" RAG-for-ASTE design.
Fairness rules apply across all systems. They share training data and retrieval store contents. LLM systems use the same model and a matched prompt-token budget. Every system gets an equal hyperparameter search budget, at most twenty configurations on the development split. Three seeds are run for every trained component.
Ablations isolate the key ingredients. The first removes retrieval (reasoning over candidates only). The second removes reasoning (direct verdicts given precedents). The third replaces polarity-balanced retrieval with plain top-k similarity. The fourth replaces index-pointing output with free span generation. Beating the baselines means two things. On DMASTE overall and on the trap subset, the full system must exceed both the tagger and the reasoning-free selector by more than twice the seed standard deviation. On SemEval, it must lose no more than that margin.
Why not just…?
Why not just fine-tune a better tagger? On short in-domain sentences this may well be enough, and we expect at best parity there. The case against stopping at the tagger is the "length penalty" reported in Measuring Your ASTE Models in The Wild: taggers degrade sharply on reviews 3.6 times longer than SemEval sentences, and they suffer negative transfer across domains. The length-bucketed and cross-domain DMASTE results settle whether the judge adds anything there. If it does not, the honest conclusion is that the tagger suffices.
Why not just let the LLM generate triplets from structure-aware retrieved examples? MiniConGTS shows that GPT-4 with CoT loses to a small tagger because it rewrites spans. Better demonstrations do not prevent rewriting. Our ablation that switches from index-pointing to free generation, with everything else fixed, measures how much of any gain comes from never letting the LLM write spans.
Why not just have the LLM select among candidates directly, as Li et al. do? This is the real threat to the proposal, because it keeps the precision benefits and is cheaper. The "no reasoning" ablation answers it directly. If direct selection with the same precedents matches the full system, especially on the trap subset, then step-by-step reasoning is not earning its cost. We would report that and recommend the cheaper design.
Why not just use RLI-style label blending, which already retrieves triplets? On the trap subset, blending neighbor labels should actively hurt, since that subset contains, by definition, neighbors with the wrong polarity. That is exactly where a judge that argues about transfer should win. The trap-subset comparison between RLI and our system is the decisive test of this claim.
Proposed Experiments
We evaluate exact-match triplet F1 on the four SemEval-derived ASTE splits (14res, 14lap, 15res, 16res, D_b versions). We also evaluate on DMASTE, both in-domain and in leave-one-domain-out transfer across its eight Amazon domains, reporting results by sentence-length bucket. Three diagnostic subsets are built automatically from test data:
- A trap subset, where a test triplet's nearest training neighbor by plain sentence similarity has the opposite polarity.
- A multi-overlap subset, with one-to-many or many-to-many pairings.
- A long-distance subset, where aspect and opinion are far apart.
Alongside F1 we report candidate recall ceilings, LLM calls and tokens per sentence, and wall-clock latency.
Three outcomes decide the proposal. Gains concentrated on the trap and long-review slices, which vanish when either reasoning or polarity-balanced retrieval is removed, would confirm the core claim. Gains that survive removing reasoning would demote CoT to unnecessary. And verdicts that do not change when precedent labels are flipped would show that the reasoning is not actually using retrieval, which would falsify the mechanism even if F1 improves.
References
- MiniConGTS: A Near Ultimate Minimalist Contrastive Grid Tagging Scheme for Aspect Sentiment Triplet Extraction
- Reasoning Implicit Sentiment with Chain-of-Thought Prompting
- Aspect-Based Sentiment Analysis with Syntax-Opinion-Sentiment Reasoning Chain
- Zero-Shot Cross-Domain Aspect-Based Sentiment Analysis via Domain-Contextualized Chain-of-Thought Reasoning
- Making Better Use of Training Corpus: Retrieval-based Aspect Sentiment Triplet Extraction via Label Interpolation
- In-Context Example Retrieval from Multi-Perspectives for Few-Shot Aspect-Based Sentiment Analysis
- Instruction Tuning with Retrieval-based Examples Ranking for Aspect-based Sentiment Analysis
- Aligning Black-Box LLMs for Aspect Sentiment Quad Prediction
- Retrieval-Augmented Chain-of-Thought in Semi-structured Domains
- ASTE-Transformer: Modelling Dependencies in Aspect-Sentiment Triplet Extraction
- Measuring Your ASTE Models in The Wild: A Diversified Multi-domain Dataset For Aspect Sentiment Triplet Extraction