Letting Explanations Veto Recommendations

When someone posts on a book forum asking for "a slow, cozy mystery, nothing gory, something like Louise Penny," the useful answer is a short list of books plus a connected paragraph explaining why each one fits. Current systems produce the two pieces separately. A ranker picks the books, and a generator writes a justification for each book independently afterwards. Nothing checks whether the stated reasons are true of the books, or whether the list as a whole honours the request. If the system writes "gentle and cozy" about a book whose readers describe it as bleak and violent, that error usually goes unnoticed. The ranking error behind it also goes unnoticed.

We propose to reverse the order of authority. The system first drafts a list-level explanation organised around the constraints in the request. It then checks every claim it makes about every book against what readers actually wrote in reviews and tags. When a claim cannot be supported, it runs a targeted follow-up search of that community text. If the evidence still does not support the claim, it removes the book and replaces it, rather than only rewording the sentence. The claim is that an explanation which must be backed by reader-written evidence is a better test of a recommendation than the ranker's score. Under this view, explanation failures become a signal that improves the list, and they stop being a cosmetic defect.

Core Claim

The proposal treats the four directions unequally. Iterative query refinement supplies the core mechanism: a draft, then verification, then revision. Social metadata (reviews, tags, crowd content warnings) is the evidence the verifier consults. Recommendation explainability is the object being produced. Global narrative coherence is the constraint that makes the problem hard. The explanation is one document covering several items, so it must stay consistent with the original request across all of them. Contradictions are measured both between items and between each item and the request.

The specific new step is this. When verification fails, the system is allowed to change which items are recommended, not only the text. We predict two things. First, this "veto" loop will raise ranking quality on forum-request benchmarks above a single-pass reranker that reads the same reviews. Second, it will sharply reduce constraint violations, meaning recommended items that break a "nothing gory"-style requirement, in cases where no structured tag reveals the violation.

Why This Gap Exists

The motivating evidence comes from Social Book Search, a series of shared tasks built on real LibraryThing forum requests. The tasks are described in Overview of the INEX 2012 Social Book Search Track, Overview of the INEX 2013 Social Book Search Track, and Overview of the INEX 2014 Social Book Search Track. These tracks establish that forum requests mix topical needs with affective and stylistic ones, which professional catalogue data cannot capture.

Social Book Search: The Impact of Professional and User-Generated Content on Book Suggestions quantifies the gap. With an Indri language-model retriever, indexing the review field reached recall 0.720, while professional subject headings reached 0.008. Human judges also said they had "not enough information" to judge recommendability in 37% of cases without reviews, compared with 1% with reviews. Reviews are therefore where the evidence for affective claims lives. The INEX 2012 overview adds a constraint on our design: elaborate personal-profile systems failed to beat plain-text retrieval over reviews. For that reason we build on review text and do not attempt richer profile modelling.

The explainability work closest to us retrieves review text but never closes a loop. EGCR: Explanation Generation for Conversational Recommendation feeds BERT-encoded IMDb reviews and knowledge-graph entities into GPT-3 to explain conversational recommendations turn by turn. RevCore: Review-Augmented Conversational Recommendation retrieves reviews whose sentiment matches the dialogue and copies phrases from them into responses. Explainable Recommendation with Personalized Review Retrieval and Aspect Learning (ERRA) retrieves other users' review sentences with vector search to avoid generic explanations. EXTRA: Explanation Ranking Datasets for Explainable Recommendation reframes explanation as ranking existing review sentences.

All four retrieve once and explain one item at a time. None of them lets a failed explanation alter the recommendation. Coherency Improved Explainable Recommendation via Large Language Model (CIER) is the one paper that treats coherence as a target. It conditions generation on a soft rating so that a five-star prediction is not paired with negative text, and reports 89.8% logical coherence on Amazon. Its notion of coherence is rating-to-text within a single item. Ours is request-to-claim and claim-to-evidence across a whole list.

The mechanism comes from retrieval-augmented question answering. Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation (CoV-RAG) scores a draft, judges whether it is reliable, and issues a revised query if it is not. It reports that revising after drafting beats rewriting the query before retrieval, because the draft reveals what is missing. CoV-RAG reached 75.1% average accuracy across four open-domain QA benchmarks. Think&Cite adds a tree search with an entailment-based attribution reward and reaches 89.5% citation recall on ASQA. Chain-of-Rewrite triggers second-pass queries from model uncertainty.

In recommendation, Diagnostic-Guided Dynamic Profile Optimization (DGDPO) closes a loop on user simulators. It needs ground-truth clicks to detect errors, which is exactly what is unavailable when answering a new forum request. Our verifier replaces that missing ground truth with a check against community text. In question answering, a failed check means the system finds a better document. In our setting, a failed check can mean that the item itself was wrong, and we test that difference directly.

Research Program

The system has four stages, and each one can be evaluated on its own.

The first stage turns the forum request into a constraint ledger. This is a short structured list of positive and negative requirements, each typed as topical, affective, stylistic, or content-avoidance, and each with an importance weight. The ledger also serves as the outline for the explanation. This borrows the plan-then-write idea of Improving Attributed Long-form Question Answering with Intent Awareness, which had models declare structural intent before writing and gained 12.3 points for 4B–8B models. Our plan is not a fixed discourse taxonomy. It is the set of user constraints, which avoids the rigidity that paper's authors acknowledge in formal intent categories.

The second stage retrieves candidates from a review-and-tag index, following the plain-text finding from INEX. A single-pass LLM reranker then orders them.

The third stage drafts one explanation for the top-$k$ list. Every sentence states which constraint it addresses and which book it is about. A verifier then checks each such claim against retrieved review and tag text, using an entailment model in the spirit of Think&Cite's attribution reward. Each claim receives one of three labels: supported, contradicted, or unsupported. The verifier also flags contradictions across items, for example calling two books "the darkest" or "the lightest" of the set.

The fourth stage revises. An unsupported claim triggers a targeted follow-up query over that book's reviews and tags, built from the claim and the constraint, in the style of CoV-RAG. A claim that remains contradicted, or that stays unsupported after a fixed retrieval budget, vetoes the book if it concerns a high-weight constraint. The next candidate is promoted and goes through the same check. The loop stops when every high-weight constraint is backed for every listed item, or when the budget is exhausted.

The analysis component asks when the veto helps. We will stratify results by constraint type and by how much review text each item has. The sparse-review regime is where ERRA reports generic explanations, and it is also where the veto could wrongly punish good but under-reviewed books.

Baselines

The naive reference point is plain-text retrieval over reviews, tags, and descriptions with no reranking and no explanation. This is the INEX-era system that profile-based methods failed to beat. Every component must improve on it. The file contains no reported nDCG@10 for this baseline on Social Book Search, so no reported result was found on Lacuna for that metric. We will re-run it ourselves on the same index. The only reported number in this family is the 0.720 review-field recall from Social Book Search: The Impact of Professional and User-Generated Content on Book Suggestions. We will reproduce it as a sanity check of our index.

The strongest competitor is a single-pass LLM reranker that reads the same review excerpts in its context, followed by a list explanation written by the same model. It has the same information as our system and the same generator, and it lacks only the verify-and-veto loop. That makes it the cleanest test of whether the loop adds anything. For fairness, it receives the same total token and call budget as the loop. We spend the extra budget on longer review context and on self-consistency voting over several rerank orderings, so that any gain we show cannot be attributed to extra compute.

Two explanation-side baselines represent the literature. The first is an ERRA-style pipeline that retrieves review sentences per item and writes per-item explanations (ERRA), concatenated into a list. The second is CoV-RAG's revise loop applied with text-only revision (CoV-RAG). The second is also our key ablation. It runs the full verification and re-retrieval loop, but when a claim fails it may only rewrite or drop the sentence, never replace the book.

If the full system beats the text-only ablation on ranking metrics and on violation rate, the veto is doing real work. If it ties, the contribution reduces to faithful explanation writing, which is a weaker but still reportable result.

Beating the strongest baseline requires two things. Ranking quality (nDCG@10 against forum-suggested books) must improve by more than the variance across three random seeds of candidate order. The constraint-violation rate must fall, and it must fall without loss on ranking quality.

Why not just…?

Why not just let a strong LLM rerank with the reviews in its context and skip the loop? This is the real threat. A long-context model that reads twenty reviews per book may already infer "this is gory" without any explicit check. The matched-budget reranker baseline exists to settle this. Our argument for the loop rests on the CoV-RAG finding that the draft exposes what is missing, which a reranker scoring candidates in isolation never sees. Examples are a cross-item contradiction, or a constraint that no retrieved excerpt addresses. If the matched-budget reranker equals the full system on violation rate, we will report that the loop is unnecessary for this task.

Why not just filter with structured content-warning tags? For avoidance constraints that a crowd database covers, a filter is close to perfect. Datasets for Navigating Sensitive Topics in Recommendation Systems joins MovieLens with Does the Dog Die? and AO3 with trigger-warning labels for exactly this reason. We therefore do not compete with the filter on its home ground. We hide the structured warnings from every system, use them only as ground truth, and report the tag-based filter as an oracle upper bound. The question we are asking is how much of that oracle the verify-and-veto loop recovers from free text alone. That matters because most affective constraints, such as "cozy," "slow-burn," or "not preachy," have no structured tag anywhere.

Why not just generate a faithful explanation per item and concatenate? Per-item faithfulness does not guarantee that the list fits the request. Each sentence can be true of its book while the list still contains three near-duplicates or misses a constraint entirely. The ERRA-style baseline measures this directly, and our cross-item contradiction and constraint-coverage metrics are designed to expose the difference.

Proposed Experiments

The primary benchmark is the Social Book Search topics from 2012–2014, with forum-suggested books as relevance labels. We compare our system against the baselines on nDCG@10. We also report a constraint-violation rate. For this, annotators label each request's ledger. A listed item counts as a violation if a held-out set of its reviews, never shown to the system, contradicts a high-weight constraint.

The second benchmark is MovieLens joined with Does the Dog Die?, and AO3 joined with trigger warnings, from Kovacs et al. We build synthetic avoidance requests from users' histories and measure how far each system falls short of the structured-tag oracle. That paper's finding that SVD already deamplifies blood/gore for 57.3% of users gives a collaborative-filtering reference point. We will also run SVD as an extra reference.

Explanation quality is measured three ways. The first is claim attribution precision: the share of claims entailed by held-out reviews. The second is cross-item contradiction rate. The third is a human study. We do not rely on LLM judges for satisfaction, because Language Models Don't Know What You Want found such judges no better than a majority baseline at predicting real user satisfaction with personalised output. Forum-style raters will compare blind pairs (our system versus the text-only ablation) on trust and on whether they would read the books.

The decisive check is the ablation, stratified by review density. The proposal succeeds if the veto loop beats text-only revision on violations and on nDCG@10 for items with moderate-to-rich reviews, and does not hurt items with sparse reviews. If it hurts sparse-review items, we will test a fallback that abstains from the veto when evidence is thin. We will report latency and token cost alongside every result, because iteration cost is the known weakness of this family of methods.