Recommender systems that explain themselves usually draw their reasons from one of two places. Some quote past behavior ("you buy this every ten days"). Others quote an idealized picture of the user that a language model infers ("you like minimalist design"). Both miss something that decades of survey research has documented. People's stated reasons and their actual choices diverge, and they diverge in predictable directions. This divergence is called hypothetical bias: the gap between what people say they will do when saying costs nothing and what they do once price, effort, or habit gets in the way. A recommender that explains with a user's own stated goals ("you said you're cutting sugar") makes a claim about future behavior. That claim can be checked against what the user does next, and nobody currently checks it.
We propose to measure, for each user and each product attribute, how far stated preferences mislead about later behavior. We would then build an explainable recommender that predicts from both sources and cites a stated reason only in proportion to how reliably that kind of statement has predicted behavior. The stated preferences come from review text that already exists in the Amazon Review 2023 data. The method borrows the two-headed stated/revealed design from travel-choice econometrics and scales it to catalog recommendation with an LLM backbone. The concrete test is whether purchases that look "illogical" given a user's history become predictable once their earlier stated aspirations are taken into account. A current method throws those purchases away as noise.
Core Claim
More Than What Was Chosen (C-APO) shows that roughly 30% of Amazon Review 2023 purchases cannot be explained from a user's prior history. It trains an LLM recommender to down-weight those purchases when an LLM judge finds a rejected item more "coherent" than the chosen one. This paper motivates our work, and it is also the closest competitor. Its coherence signal is built entirely from past behavior, so it cannot tell an accidental click apart from a purchase the user had announced wanting to make, such as a reviewer who wrote "trying to switch to unscented products" and later buys one.
Our claim has three parts. First, a measurable share of these incoherent purchases is predicted by attribute-level intentions the user stated earlier in review text. Second, the reliability of stated preferences varies systematically by attribute. Health and ethical attributes should be overstated relative to behavior, while price and convenience should be understated. Discrete choice studies report exactly this pattern: out-of-pocket cost overrides stated priorities in NSCLC insurance preferences, and logistics suppress action despite favorable attitudes in elderly COVID-19 vaccination. Third, a recommender that models stated and revealed signals with separate output layers and a learned gap between them will predict future interactions better and produce explanations whose cited reasons are better grounded than either a revealed-only model or a model that simply reads review text as extra input.
Research Program
The first step builds a stated-preference layer for the Amazon Review 2023 domains that C-APO already uses: Grocery, Health, Fashion, Clothing, and Scientific. An LLM extractor tags each review sentence that expresses a forward-looking or attribute-level intention ("I need gluten-free," "next time I'll get the cheaper one," "I only buy organic"). It maps each sentence to catalog attributes taken from item metadata, and each tag carries a timestamp. We hand-label a few hundred sentences to measure extractor precision. The revealed side is the user's subsequent interactions and the attributes of those items.
One caveat must be designed around. Amazon reviews record only reviewed purchases, so stated repurchase intent is rarely observable. We therefore restrict statements to attribute-level intentions, which can be checked against later interactions with any item, and we treat the reviewed-purchase selection effect as a known limitation.
The descriptive analysis comes next. It estimates, per attribute and per user, the probability that a stated intention is followed by matching revealed choices within a window. It also measures how much of C-APO's incoherent 30% is covered by prior stated intentions. That number is the first decisive quantity in the study. If it is near zero, stated text adds nothing that history lacks, and the modeling stage is not worth building.
The model follows the logic of Multitask Learning Deep Neural Network to Combine Revealed and Stated Preference Data (MTLDNN, also journal version). That work fit shared layers with separate stated and revealed output heads, soft-tied the heads' weights, and put a temperature on the stated head to absorb the extra randomness of hypothetical answers. On travel mode choice, this design beat nested logit by about 5% accuracy, and the same paper found that an unconstrained deep network without those ties did worse than logit. That negative result constrains our design: the tie and the temperature are not optional. MTLDNN, however, handled five alternatives and produced elasticities rather than item rankings or explanations.
We transplant the structure onto the C-APO backbone, gemma-3-4b-it, using the released C-APO codebase. Concretely, we train with
$$ \mathcal{L} = \mathcal{L}_{\text{RP}} + \alpha\,\mathcal{L}_{\text{SP}}(T) + \lambda \sum_a \lVert w^{\text{RP}}_a - w^{\text{SP}}_a \rVert^2 . $$
Each term is defined as follows. $\mathcal{L}_{\text{RP}}$ is the next-interaction preference loss on observed choices. $\mathcal{L}_{\text{SP}}(T)$ is a loss for predicting the attributes a user states they want, with the stated head's logits divided by a learned temperature $T$. Each $w_a$ is a head's weight on attribute $a$. $\alpha$ and $\lambda$ trade off the terms. After training, the per-attribute difference $w^{\text{SP}}_a - w^{\text{RP}}_a$ is our estimate of hypothetical bias for that attribute.
At explanation time, the LLM receives an evidence card in the style of Beyond Ranking Accuracy. That paper showed LLM rationales are far better grounded when raw numbers are converted to user-relative percentiles. Each card lists the user's stated intentions together with their estimated reliability. The model is instructed to say "you said X, and you've followed through on X before" only when reliability is high, and to fall back on behavioral or feasibility reasons otherwise. The goal is to replace C-APO's down-weighting of conflicts with an explicit model of the conflict.
Baselines
The naive reference point is Personal Top, which ranks items by the user's own purchase frequency and recency, plus global popularity for new items. Beyond Ranking Accuracy reports that Personal Top beat GPT-4o and Gemini 2.5 Pro on NDCG and Recall in repurchase settings. It costs nothing to run and anchors every table. Beating it requires better Recall@10 and NDCG@10 on held-out future interactions, and above all on the incoherent subset, where it should be weakest.
The strongest existing method is C-APO, run from its released code on the same domains, splits, backbone, and GPU budget. Its tuning budget, measured in the number of hyperparameter configurations, will match ours. On reported numbers, C-APO achieved a 1.65× relative CTR improvement in an online production A/B test, and GPT-4o rated its explanations "Persuasive" 84.33% of the time, both in More Than What Was Chosen. Neither figure can be reproduced offline. For offline Recall or NDCG on Amazon Review 2023, no reported result was found on Lacuna for C-APO, so we will rerun it ourselves rather than quote a number.
Beating C-APO means three things. The first is a statistically significant gain on future-interaction Recall@10 for the incoherent subset, with no loss on the coherent subset. The second is higher grounding under the cross-model masking protocol. The third is better calibration of cited stated reasons, defined below.
The second strong competitor is XGBoost with TreeSHAP attributions. It uses the behavioral features from Beyond Ranking Accuracy plus review-derived stated-intention features. That paper found XGBoost beat LLMs on ranking, and that GPT-4o's cited features matched TreeSHAP's explanatory power. A cheap tree model with the same stated features is therefore a serious threat. It gets the same features, the same split, and a matched tuning search.
Three ablations isolate the key ingredient. The first is a single-head model that receives the review-derived intentions as plain input text. This tests whether separate heads matter or whether the information alone matters. The second is the two-head model with $T$ fixed at 1 and $\lambda$ set to 0. This tests whether the MTLDNN constraints carry over to catalog scale, as they did for travel. The third is our model with the reliability column removed from the evidence card. This tests whether explanation quality comes from the calibration itself or merely from mentioning stated goals.
Why not just…?
Why not just put the review text into C-APO's prompt? This is the real threat. Feeding stated intentions as context may recover most of the ranking gain with no architectural change, and the single-head ablation tests exactly that. If it matches us on ranking, the claim narrows to explanations. A single-head model never produces a per-attribute estimate of how much stated preferences mislead, so it has no principled way to decide when to cite "you said." The calibration metric below would then decide the question. If the single head also matches on calibration, we would report that the separate-head design is unnecessary.
Why not just use XGBoost and TreeSHAP with stated-intention features? On ranking it may well win, given its record in Beyond Ranking Accuracy. The same paper notes, however, that raw attributions are not rationales a customer can read. We would therefore accept a tree ranker as the scoring model if it wins and attach our reliability-aware explanation layer to it. That would be an honest and still useful outcome. The scientific claim about stated-revealed gaps does not depend on which ranker scores the items.
Why not just fit a nested logit or the original MTLDNN? Nested logit requires hand-specified utilities over small choice sets. MTLDNN's own comparison showed it underperforms the multitask network by about 5% even at five alternatives (Wang and Zhao). We will still fit a logit on a sampled 5-candidate version of our task as an interpretability sanity check. If the per-attribute gaps it recovers agree with ours in sign, the neural estimates gain credibility.
Why not just evaluate explanations with an LLM judge, as C-APO does? Language Models Don't Know What You Want found that LLM judges missed nine failure modes that real users reported, and that no judge predicted human satisfaction better than a majority-class baseline. A persuasiveness rating from a judge is itself a costless stated reaction, which is the kind of measurement hypothetical bias corrupts. We use LLM judges only as a secondary signal and test their agreement with behavior directly.
Proposed Experiments
The first decisive check is descriptive. We measure what fraction of C-APO-style incoherent purchases are preceded by a matching stated intention, compared against a permutation baseline in which intentions are shuffled across users. We also estimate per-attribute follow-through rates and test the predicted ordering: health and ethical attributes overstated, price understated.
The second check is predictive. All baselines and ablations are evaluated with Recall@10 and NDCG@10 on chronologically held-out interactions, reported separately for coherent and incoherent targets and for the zero-shot Beauty and CDs domains that C-APO used.
The third check concerns explanation grounding. We apply the feature-masking protocol of Beyond Ranking Accuracy to the reasons each model cites, including stated-intention features. We also introduce a said-did calibration score: the Brier score of the reliability that an explanation implicitly asserts when it cites a stated reason, scored against whether the user's next interactions in a fixed window match that attribute.
As a secondary, friction-oriented replication, we use GLEN-Bench. Its NHANES questionnaire answers about diet goals serve as stated preferences, dietary recall serves as behavior, and Purchase-to-Plate prices together with poverty-income ratio test whether economic friction explains the gap. Both of its signals are self-reported, so we treat this setting as supporting evidence only.
Finally, given the simulation trap, we run a small incentive-compatible user study. Participants first state their preferences and then choose from a catalog with a real budget, half seeing reliability-aware explanations and half seeing C-APO-style explanations. We record stated trust in the explanations and actual choices. Alongside this, we measure whether GPT-4o persuasiveness ratings predict either outcome.
The project fails cleanly in two ways. Stated intentions may cover no more incoherent purchases than shuffled intentions do, or the single-head text-input ablation may match the full model on both ranking and calibration. Either result would itself show that, for consumer recommendation, the stated-revealed gap carries no exploitable structure beyond what behavior already reveals.
References
- More Than What Was Chosen: LLM-based Explainable Recommendation Beyond Noisy User Preferences
- Preferences for public health insurance coverage of new anticancer drugs: a discrete choice experiment among non-small cell lung cancer patients in China
- The COVID-19 vaccination decision-making preferences of elderly people: a discrete choice experiment
- Multitask Learning Deep Neural Network to Combine Revealed and Stated Preference Data
- Multitask learning deep neural networks to combine revealed and stated preference data
- Beyond Ranking Accuracy: Evaluating LLM-Cited Feature Rationales for Next Basket Repurchase Recommendation
- Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users
- GLEN-Bench: A Graph-Language based Benchmark for Nutritional Health