Fine-Grained User Affect as a Label-Efficient Signal for Monitoring Trust in Deceptive Dialogue

When a conversational AI agent tries to manipulate a user, through a fraud script, a pushy product pitch, or emotional pressure, the harm usually comes from how much the user ends up trusting the agent. The agent's intent is only part of the story. A safety monitor should therefore track the user's trust turn by turn and intervene before it crosses a dangerous level. The best existing attempt, TrustNet in OpenDeception, learns user trust directly from dialogue text and reaches a little over 77% accuracy. It needs more than 61,000 hand-built contrastive pairs to get there. Labels for trust are scarce and noisy, while labels for emotion are comparatively plentiful.

We propose a trust monitor that reads the user's side of the conversation through a fine-grained emotion layer. That layer separates states such as polite compliance from genuine agreement, or embarrassment from reassurance-seeking, in the same way a polite smile differs from a real one. The monitor then predicts trust from those states plus the text. We claim that this explicit, finer-than-basic-emotions layer makes trust monitoring far more label-efficient and makes it transfer better to deception scenarios it was not trained on. We also claim that disagreement between the emotion-based and text-based views of trust is a good way to choose which dialogue turns humans should label.

Core Claim

Psychology already suggests that trust tracks emotion in a specific way. The Influence of Emotion Expression on Perceptions of Trustworthiness in Negotiation ran agents with fixed negotiation strategies and showed that a generic positive display did not earn trust. Smiling raised partner choice for cooperative tit-for-tat agents by 25% over an angry display. For selfish agents the angry display was preferred by 25%, and a smiling selfish agent read as deceptive. What mattered was whether the emotion matched the behavior.

That study varied the agent's emotions with crude discrete toggles. We turn the observation around and apply it to the user. Whether a user's expressed affect is consistent with their stated compliance should be an early, readable sign of where their trust is heading. Coarse labels cannot express this. "Joy" covers both a user who is genuinely delighted by an offer and one who is politely going along while uneasy. This mirrors how Fine-Grained Facial Expression Recognition in Multiple Smiles showed that collapsing six socially distinct smiles into "happiness" throws away recoverable information (88.56% accuracy over a Swin baseline at 80.03% once the categories are split).

The claim to be tested is specific. On the OpenDeception scenarios, a trust monitor with an explicit fine-grained user-affect layer should reach TrustNet-level pairwise trust accuracy with a fraction of the trust labels. It should also degrade less when evaluated on a held-out deception domain. Both advantages should shrink or vanish when the affect layer is replaced by coarse basic-emotion labels.

Research Program

The monitor is built as three separable parts with fixed interfaces. They are kept separate so that each scientific question becomes a clean swap rather than a re-engineering effort, following the factory-style decoupling of FaceX-Zoo. The monitor also sits entirely outside the conversational agent. That choice is motivated by OpenDeception's finding that more capable models deceive more, with three reasoning models near 100% deception intent. Asking the agent to monitor itself is therefore not credible. The cascade evidence in AHELM, where a transcription-plus-LLM pipeline ranked 6th of 17 audio-language systems and led on noise robustness, tells us that a modular cascade need not cost accuracy. We still test that claim rather than assume it.

The first part is a user-affect module. It is a LoRA-tuned 7B model in the style of DialogueLLM, meaning a large language model adapted with small low-rank weight updates and given dialogue context. It is pretrained on MELD, IEMOCAP and EmoryNLP with their coarse labels.

The affect module is then extended to a fine-grained inventory of user-side social-affective states. We build this inventory by splitting coarse classes along social function, as Jin et al. split smiles. Joy splits into genuine enthusiasm, polite compliance and relief. Fear splits into wariness, pressured urgency and reassurance-seeking. Further additions are embarrassment, guilt, flattered susceptibility and hesitation. We label a few thousand user turns sampled across the five OpenDeception domains with multiple annotators per turn. We keep the annotators' soft label distributions rather than forcing agreement, because disagreement grows as categories get finer. The module outputs a per-turn probability vector over this inventory.

The second part is the trust head. It is a small sequence model that takes the dialogue text representation and the affect vectors up to turn $t$ and outputs a continuous trust score. It is trained with TrustNet's contrastive objective, which ranks the higher-trust response above the lower-trust one in matched contexts, because OpenDeception found absolute trust labels unreliable. The continuous score feeds a risk-adjusted threshold policy in the spirit of Using Artificial Intelligence Techniques to Implement a Multifactor Authentication System.

The threshold policy works like usage control, the runtime downgrading of permissions described in Security and Trust in Digital Rights Management: A Survey. Suppose user trust rises while an intent detector such as OpenDeception's IntentNet flags the agent. The session then loses specific capabilities, for example requesting payment details or personal identifiers, rather than being killed outright.

The third part is the labeling-selection rule. Because the trust head can be trained in two views, affect-only and text-only, we pick unlabeled dialogue pairs where the two views most disagree about which response reflects higher trust. This is a two-view form of query-by-committee, a standard active-learning family. Its specific bet is that cases where the user's affect and their words point in different directions are exactly the action-consistency violations Antos et al. found to be diagnostic. A batch-level diversity term, following the furthest-from-centroid rule of O-MedAL, prevents the rule from repeatedly selecting near-duplicate pairs from one scenario.

Baselines

The naive reference point is a monitor that does no learning from the user at all. It flags a session only from the agent side, using IntentNet's deception-intent signal, and assigns every user a fixed prior trust. This establishes how much tracking the user adds beyond watching the agent. We also report majority-class and random pairwise accuracy so that the 77% figure has context.

The strongest existing method is TrustNet from OpenDeception, a contrastive text-only user-trust model that reports over 77% accuracy on held-out contrastive pairs. We will retrain it ourselves on the same backbone size, the same training pairs and the same hyperparameter-search budget as our monitor. Any gain will then come from the affect layer, not from a bigger model or more tuning.

Beating TrustNet means one of two things. Our monitor either matches its full-data pairwise accuracy using at most one third of the trust-labeled pairs, or it exceeds it by a margin larger than seed-to-seed variance on the held-out-domain split. A gain only on the in-domain full-data split would not count, because a text model could plausibly close that gap.

A zero-shot LLM judge is a third baseline. A strong instruction-tuned model is prompted to compare two user responses and say which reflects more trust. It requires no labels, which makes it the natural competitor at the very low end of the label-efficiency curve.

Two ablations isolate the key ingredient. The first keeps the full architecture but feeds the trust head coarse seven-class affect from the same DialogueLLM-style module trained only on MELD, IEMOCAP and EmoryNLP. This tests whether granularity, not merely the presence of an affect layer, drives any effect. For reference, DialogueLLM reports 71.90% F1 on MELD, so the coarse module is competitive. The second gives the trust head a frozen generic sentence embedding of equal dimension in place of the affect vector. This rules out the explanation that any extra pretrained feature helps.

For the selection rule, the competitors are random sampling, length-normalized uncertainty sampling as in Deep Active Learning for Named Entity Recognition, and two-model query-by-committee where both committee members are text-only models with different seeds. The last comparison isolates whether the affect view specifically makes disagreement informative. Every strategy gets the same labeling budget per round, the same number of rounds and the same seed pool. No reported active-learning result on trust prediction was found on Lacuna. The closest data-efficiency reference points are from other tasks: 99% of peak with 25% of data in Yun et al., and full-data performance surpassed with 35% of labels by the competency curriculum in Feeding What You Need by Understanding What You Learned.

Why not just…?

Why not just train TrustNet on more data, or with a larger text model? A large enough text model may learn fine-grained affect implicitly, and this is the main threat to the proposal. If that happens, the explicit layer is just an inductive bias that more data washes out. We do not argue around this. The label-efficiency curve settles it. If TrustNet trained on all 61,000 pairs is matched by our monitor at a third of the labels but beats it at full data, the contribution narrows to low-label and transfer regimes. If TrustNet matches our monitor at every budget and on the held-out domain, the core claim is false and we will report that.

Why not just use coarse basic emotions, or valence and arousal? These have mature recognizers and benchmarks. However, both the smile results of Jin et al. and the action-consistency results of Antos et al. indicate that the trust-relevant distinction lives inside a coarse category. Work on dimensional models such as Parada-Cabaleiro et al. argues that two axes cannot carry it. Still, the coarse-affect ablation is the experiment that decides this. If it recovers most of the gain, fine-grained annotation is not worth its cost, and we would recommend the cheaper coarse layer.

Why not just ask a strong LLM to rate user trust zero-shot? It needs no labels and may already encode social nuance. Its known weakness here is calibration. OpenDeception moved to contrastive training precisely because absolute trust judgments were unreliable. We include it as a baseline, scored on the same pairwise task and on per-turn early-warning lead time. If it matches the trained monitor, the practical recommendation becomes "prompt, don't train," and the affect layer would need to justify itself on latency or cost instead.

Why not just use plain uncertainty sampling for choosing labels? It is simple and strong across tasks, and it may well capture most of the selection benefit. The comparison against uncertainty sampling and against text-only query-by-committee shows whether the affect view adds anything to selection. If it does not, the selection rule is dropped from the contribution, and the affect-layer claim stands or falls separately.

Proposed Experiments

The primary testbed is the OpenDeception benchmark of 50 scenarios across telecom fraud, product promotion, personal safety, emotional deception and privacy stealing, together with its contrastive user-response pairs. The metric is pairwise trust-ordering accuracy, matching TrustNet's reported metric.

The first decisive check is the label-efficiency curve. We train our monitor, TrustNet and the ablations on 5%, 10%, 20%, 35% and 100% of the trust-labeled pairs. We report the label fraction at which each method reaches TrustNet's full-data accuracy, averaged over five seeds.

The second check is cross-domain transfer by leave-one-domain-out. The emotional deception domain is the critical held-out case, because it is where our hypothesis predicts the largest gap. The prediction is that affect-mediated trust transfers while text-only trust overfits to scenario vocabulary.

The third check is early warning. We replay full sessions and measure how many turns before the user's trust crosses a fixed threshold each monitor raises an alarm. We also measure the false-alarm rate on sessions with benign agents. The point is to test whether a better ranking actually translates into earlier, usable intervention under the usage-control policy.

OpenDeception's users are simulated by language models, whose expressed affect may be stereotyped in ways that flatter an affect-based monitor. We therefore add a small human validation study. Participants converse by text with scripted agents of varying deceptiveness. They then make an incentivized investment decision following the trust-game protocol of Antos et al., which measures behavioral trust rather than self-report. They also give a per-turn trust slider rating.

The check on this human data is whether our monitor's per-turn score predicts the final investment better than TrustNet's score does. We measure this by rank correlation across participants. If the simulation-trained affect advantage disappears on real people, the right conclusion is that fine-grained affect helps only in simulated dialogue. That result is worth reporting, and it would redirect effort toward real user data.