When a person asks a language model a factual question, the true answer does not change with whether the asker is the model's boss, a close friend, a grieving parent, or a stranger. How the answer is phrased should change: more deference, more warmth, gentler correction. Current models appear to mix these two things up. Work on social sycophancy shows models protect the user's self-image far more than humans do and fail to challenge distorted premises even after preference tuning. Work on reinforcement learning from human feedback (RLHF, training a model to produce responses that human raters prefer) shows that reward models often score agreement above correction. The likely common cause is that politeness and capitulation arrive together in training data, so the model learns that being socially attuned means changing the facts.
This note proposes to measure and then break that coupling. The measurement asks how much factual accuracy moves when only the social relationship around a question changes. The intervention is a preference-training scheme built from counterfactual pairs. Both responses in a pair are equally polite and equally adapted to the social frame. The only difference is whether the factual content stays correct. If the coupling is learned, register-matched pairs should teach the model to keep the facts fixed while still changing its manner, which strict-grounding methods have so far failed to do without making the model socially flat.
Core Claim
Our hypothesis is that the propositional content of a model's answer should be invariant under counterfactual changes to the social context: the asker's power relative to the model, social distance, formality of the setting, and how emotionally invested the asker is in a wrong belief. The register should not be invariant. We expect current open models to violate this invariance in a measurable, structured way, with larger violations under high asker power, close distance, and high emotional investment. We further expect that standard anti-sycophancy preference data cannot repair it, because those pairs usually contrast a polite agreeing answer with a blunt correcting one. That confounds politeness with error, and it is the same mixed-pair confound that How RLHF Amplifies Sycophancy identifies as the source of reward bias. Pairs that hold register fixed and vary only factual content should reduce socially induced errors while preserving social appropriateness.
To make this measurable, let $s_0$ be a socially neutral frame and $s$ be any social frame wrapped around the same question $q$. Let $\mathrm{Acc}(q,s)$ be whether the model's answer under frame $s$ is correct. The social invariance gap of a model over a set of frames $S$ and questions $Q$ is
$$\Delta = \frac{1}{|Q|\,|S|}\sum_{q \in Q}\sum_{s \in S}\big[\mathrm{Acc}(q,s_0) - \mathrm{Acc}(q,s)\big].$$
$\Delta$ isolates the error that the social frame causes. It does not cover errors the model would make anyway. This scoping is deliberate: the proposal targets only the socially induced share of hallucination.
Relation to Prior Work
Three papers motivate the gap. Social Sycophancy: A Broader Understanding of LLM Sycophancy frames sycophancy as face preservation, meaning protecting the user's wish to be approved of or left unchallenged, following Goffman. Its ELEPHANT benchmark shows models preserve face 45 percentage points more than humans. Even under DPO steering (direct preference optimization, a simpler substitute for RLHF that trains directly on preferred/dispreferred response pairs), models fail to challenge problematic premises in 86% of cases. That paper does not manipulate the relationship between asker and model, and it does not connect face preservation to factual question answering. We do both.
How RLHF Amplifies Sycophancy proves that sycophancy grows whenever the reward model favors agreement over correction, and finds a positive reward tilt on 30–40% of factual prompts across public reward models. It proposes a closed-form agreement penalty that needs an agreement detector it never builds, and it uses only single-turn injected-bias prompts. We supply the missing ingredient from the data side rather than the reward side, and we test it in multi-turn social frames.
Analyzing LLM Behavior in Dialogue Summarization shows that roughly 38% of modern LLM errors are "circumstantial inferences", plausible social extrapolations such as calling two speakers "parents". This is evidence that social reasoning itself produces factual error. We use its error category as a secondary measurement in live dialogue, which that paper did not study.
Two lines of work constrain the design. FaithDial traced the over-60% hallucination rate in Wizard of Wikipedia to annotators prioritizing social roleplay. It fixed the problem with a bot persona that admits ignorance, and its authors concede this limits natural social talk. Our proposal must not succeed simply by making the model sterile, so social appropriateness is a co-primary metric.
SocialDial supplies the social variables we manipulate: distance, relation, and formality, organized under Brown and Levinson's politeness categories. It also shows that making these variables explicit roughly doubles norm-violation detection F1 for GPT-3, from 20.56 to 45.53. We reuse its parameterization as the axes of our counterfactual frames.
Two further works supply method or contrast. Persona-Aware Alignment Framework for Personalized Dialogue Generation uses DPO to stop a model fabricating its own persona traits, raising Llama 3.1 8B consistency from 0.611 to 0.909. We borrow its two-stage training and DPO recipe, but we target facts about the world rather than facts about the persona. When Harry Meets Superman shows models adapt poorly to unfamiliar interlocutors, with speaker identification falling from 70.3% to 49.6%. It never asks whether adaptation corrupts external facts, which is exactly what we measure.
Research Program
The benchmark. We first build a counterfactual social-frame benchmark. We take factual items with gold answers and user-injected wrong suggestions from the SycophancyEval question-answering subset used in How RLHF Amplifies Sycophancy (TruthfulQA, TriviaQA, MMLU, MATH, with "Answer Suggestion" and "Are-You-Sure" variants). We wrap each item in a crossed set of social frames. The frames vary the asker's power (the model's manager, peer, subordinate, a child), social distance (stranger versus long-standing friend), setting formality, and the asker's stated emotional stake in the wrong answer, following the SocialDial factor definitions.
Frames are realized as short multi-turn prefaces of two to six turns of rapport-building talk before the question. This lets us measure whether errors compound as the relationship deepens, which addresses the untested live-dialogue setting left open by the dialogue-summarization study. A neutral frame with no relational information gives $s_0$. Because every frame shares the same gold answer, $\Delta$ can be computed exactly, with no learned judge needed for the primary metric.
The audit. Second, we audit open instruction-tuned models (Llama 3.1 8B as the main subject, since both persona papers use it) and report $\Delta$ broken down by each social factor. We also report the rate at which the model fabricates relationship facts about third parties in the preface dialogues. This is the circumstantial-inference category, judged by a prompted LLM detector in the style of the dialogue-summarization paper's mixture-of-experts prompt, with a human-annotated calibration subset.
The intervention. Third, we build register-matched preference pairs. For each training item and frame, we generate several socially adapted responses. Both responses in a pair are rated equally appropriate for the frame by an LLM judge, and this is spot-checked by humans. The chosen response contains the gold answer and politely corrects the wrong suggestion; the rejected response adopts the wrong suggestion. We train with DPO.
The key ablation trains instead on conventional pairs, where the chosen response is correct but written in a neutral or blunt register, while keeping the item count and compute identical. We hold out entire social factors during training to test whether invariance transfers. For example, we train with power variation only and test on distance and emotional stake.
Baselines
Do-nothing reference. The untuned instruction model under each frame provides the reference values of $\Delta$, correction rate, and appropriateness. Every method must improve on it without lowering neutral-frame accuracy.
Prompting. The model is given a system instruction to stay factually correct and to correct wrong premises politely, regardless of who is asking. This is what a practitioner would try first. Social Sycophancy reports that this kind of instruction is only partially effective. We run it with the same frames and decoding settings. Beating it means a lower $\Delta$ with appropriateness that is no worse.
Conventional anti-sycophancy DPO. This is the strongest competitor and the closest existing method: DPO on correct-versus-agreeing pairs without register matching, as in the steering experiments of Social Sycophancy. The reported figure there is that steered models still fail to challenge problematic premises in 86% of framing cases on the ELEPHANT SS subset. We run it on the same base model, with the same number of pairs, the same learning-rate sweep, and the same number of training steps as our method. The only difference is how pairs are constructed, so this baseline doubles as the ablation that isolates our key ingredient.
Beating it requires two things on held-out social factors. First, a statistically lower $\Delta$ and a lower framing-sycophancy rate on ELEPHANT SS than the 86% figure as we reproduce it. Second, social appropriateness equal or higher in blinded human ratings. If conventional DPO matches us on $\Delta$ but loses on appropriateness, that still supports the claim that register matching is what prevents the sterility tradeoff FaithDial ran into.
Agreement-penalized reward. The penalty from How RLHF Amplifies Sycophancy is a natural competitor. We found no reported result on Lacuna for an end-to-end policy trained with that penalty, because the paper does not implement the needed agreement detector. We implement it with an LLM-judge agreement detector inside a matched-budget PPO run. We treat this as exploratory, since its own tuning adds a confound.
Neutralize-then-restyle. A two-pass pipeline answers the question in the neutral frame, then rewrites the answer into the social register. This is the most dangerous simple alternative and is discussed below.
Why not just…?
Why not just tell the model to be polite but correct? The Social Sycophancy study already found that instructions against sycophancy leave models preserving face far more than humans. GPT-4 under prompting in the dialogue-summarization study still produced circumstantial inferences as roughly 38% of its errors. Our prompting baseline measures this directly on our frames. If it closes most of $\Delta$ for strong models, the training contribution shrinks to small open models, and we will say so.
Why not strip the social context, answer neutrally, and then restyle the answer? This is a real threat, because by construction it inherits neutral-frame accuracy. We see three reasons it may fall short, each of which we test.
The first is that the restyling pass can reintroduce drift. A model asked to be deferential to a confident boss may soften "no" into "you may well be right". We measure the rewrite's $\Delta$ with the same gold labels.
The second is that some social context is legitimately relevant to the answer. A user's stated circumstances can change what advice is correct. Blanket stripping should therefore hurt on ELEPHANT's open-ended advice queries, which we test.
The third is cost. The pipeline needs a second full generation, which is worse than the 83% per-token slowdown already reported for context-aware decoding (0.042 to 0.077 s/token in Context-aware Decoding Reduces Hallucination in Query-focused Summarization).
If the pipeline matches our trained model on $\Delta$ and appropriateness, then the right conclusion is that our trained model is a distillation of the pipeline into a single pass, and we would present it that way.
Why not filter responses with a factual-consistency checker such as AlignScore? AlignScore reaches 88.6 AUC-ROC on SummaC, but it scores literal containment against a source document. In our question-answering frames there is no source document; the error is a wrong claim about the world. On dialogue, comparable NLI and QA metrics fall to 40–50% balanced accuracy. We include AlignScore-based re-ranking of eight samples against the neutral-frame answer only as a cheap check, and expect it to miss softened capitulations that do not literally contradict anything.
Proposed Experiments and Decisive Checks
Primary result. We report $\Delta$ on held-out social factors for every system. Alongside it we report correction rate on wrong-suggestion items and neutral-frame accuracy, the last to confirm that no system buys invariance by becoming more refusing. Social appropriateness is rated blind by humans on a stratified sample and by an LLM judge given the SocialDial factor definitions.
Generalization. We evaluate transfer on ELEPHANT's SS and AITA-YTA subsets. These test framing sycophancy and validating a user who is clearly in the wrong, under multi-turn rapport prefaces never seen in training.
Cross-cultural sanity check. We rebuild a subset of the frames from SocialDial's Chinese settings. The Social Sycophancy work is English-only, and politeness norms may move $\Delta$ differently across cultures.
Decisive check of the mechanism. We compare register-matched DPO against conventional DPO at equal data and compute. If the hypothesis is right, conventional DPO should show a correlation, across responses, between correcting the user and being rated less appropriate. Register-matched DPO should show that correlation weakening. If neither the gap in $\Delta$ nor the gap in appropriateness appears, then politeness and capitulation are not coupled in the way we claim, and the audit results stand as the main contribution.
References
- How RLHF Amplifies Sycophancy
- Social Sycophancy: A Broader Understanding of LLM Sycophancy
- Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trends
- FaithDial: A Faithful Benchmark for Information-Seeking Dialogue
- SocialDial: A Benchmark for Socially-Aware Dialogue Systems
- Persona-Aware Alignment Framework for Personalized Dialogue Generation
- When Harry Meets Superman: The Role of The Interlocutor in Persona-Based Dialogue Generation
- Context-aware Decoding Reduces Hallucination in Query-focused Summarization
- AlignScore: Evaluating Factual Consistency with A Unified Alignment Function