When a disaster, pandemic, or policy shock hits, scammers move within hours: COVID-19 policy announcements were followed immediately by matching lures, and cryptocurrency abuse reports rose thirteen-fold during the pandemic. The new attack surface is not a static phishing page but a conversation — an AI-driven agent that builds rapport, manufactures urgency, and steers a victim toward a spoofed crisis-relief portal. Large language models are the natural candidates to defend people here, yet the evidence says they fail in exactly the way human victims do: across multi-round fraud dialogues, defense success rates among fifteen evaluated LLMs ranged from 38.92% to 92.55% on Fraud-R1, and they collapse further when the model is placed in a conversational persona rather than a neutral assistant role. Existing multimodal defenses are single-shot: they inspect one URL or screenshot and return a verdict, with no way to track what an attacker is doing across turns.
This proposal builds a defender that works the other way around. Instead of asking a model to judge an ongoing, emotionally loaded conversation in one pass — the setting where LLM reasoning is demonstrably weakest — the agent decomposes each conversational turn into discrete, checkable claims ("this is the municipal relief office," "the deadline is 24 hours," "this link is the official portal"), verifies each claim in a clean context window against cross-modal evidence (live domain registries, rendered screenshots, a dynamically constructed crisis-entity graph), and replans its threat hypothesis when checks fail. The core bet has three legs: that persuasion text is a poison for the defender's own reasoning and must be quarantined from the verification steps; that crisis scams impersonate ephemeral entities (relief programs, new stimulus acts) that no static brand database contains, so reference knowledge must be built live; and that a hypothesis–verification–replanning loop, proven in scientific diagnosis but never tried under adversarial deception, is what converts a brittle one-shot judgment into a stable multi-turn one.
Core Claim
A defensive agent can hold up under multi-round, crisis-themed, multimodal social engineering if and only if three conditions are met together: the attacker's persuasion-laden text is stripped out of the reasoning context before any verification step (borrowing the error-bounding argument of Divide-and-Conquer prompting); every factual claim is checked against retrieved cross-modal evidence rather than the model's own memory; and failed checks trigger replanning rather than being averaged away. Each condition has partial support in the literature — but never in combination, and never against an adversary. Divide-and-Conquer (DaC) prompting shows that isolating sub-tasks in clean contexts bounds compounding errors, achieving 76.92 F1 on GPT-4 five-digit multiplication versus 69.41 for Tree-of-Thoughts, but it has only been tested on arithmetic, summarization, and claim verification, not under adversarial persuasion. HVR-Met demonstrates that a Hypothesis-Verification-Replanning loop with a vision-language "Image Checker" can pass 79.52% of semantic-integrity checks in extreme-weather diagnosis, but only on clean scientific data that never lies back. PhishAgent shows that cross-modal re-extraction (falling back to screenshots when DOM text is obfuscated) bounds recall loss to 4% where single-modality pipelines lose up to 20%, but it classifies one static page per call and knows nothing about time, crises, or conversations. The claim under test is that these three mechanisms compose into a defender whose success does not decay across turns — and that removing any one of the three reintroduces the decay.
Research Program
The artifact to build is a Crisis Verification Agent with four components. First, a claim extractor that maps each incoming turn (message text, embedded link, attached image, or voice transcript) into a small set of typed predicates: identity claims ("I am X"), temporal-pressure claims ("act within 24 hours"), resource claims ("funds are available via this portal"), and consistency claims about prior turns. Second, a quarantined verification layer: each predicate is checked in a fresh context window containing only the predicate and tool outputs, never the attacker's framing — this is the DaC-style context hygiene, and it is the direct answer to the finding that role-play personas collapse LLM defensive reasoning. Third, a live crisis-entity graph: unlike the 20,000-brand static base used by KnowPhish and PhishAgent, which is populated from Tranco rankings and Wikidata and therefore contains no ephemeral relief programs, the graph is constructed at runtime from news feeds, government policy announcements, and domain-registration lookups — mirroring how attackers themselves time campaigns to policy announcements. Fourth, a replanning controller that maintains competing hypotheses (for example, "legitimate relief disbursement" versus "crisis credential harvesting") and re-runs targeted checks when a verification failure contradicts the leading hypothesis, rather than terminating or averaging.
The scientific questions are as concrete as the engineering. Does context quarantine measurably protect the defender against urgency and authority framing — that is, does it close the assistant-versus-persona defense gap that Fraud-R1 documented? Can ephemeral-entity resolution reach usable precision when the reference set must be built in hours, not curated over years? And does the replanning loop pay for its latency: PhishAgent reports ~2.25 seconds per sample for single-page classification, and a multi-check agent will be slower per turn, so the proposal explicitly budgets latency per conversation turn rather than per page and treats an honest latency–robustness curve as a first-class result.
One small formalism sharpens the mechanism. Let $c_t$ be the set of checkable claims extracted at turn $t$, and let $v(c) \in \{+1, 0, -1\}$ denote the verdict of the verification layer for claim $c$ (confirmed, unverifiable, contradicted). The agent's exploitation score at turn $t$ is
$$S_t = \sum_{i \le t} \sum_{c \in c_i} \alpha_i \cdot \max(0, -v(c)) + \beta \cdot U_t,$$
where $U_t$ counts unmatched temporal-pressure claims and $\alpha_i$ weights older turns less, so a single failed check does not permanently condemn a benign correspondent. An alert fires when $S_t$ crosses a threshold; the evaluation metric of interest is how many turns before an eventual exploit the alert fires.
Proposed Experiments
Three evaluation tracks, all on material present in the literature. Track one is Fraud-R1 itself (8,564 multi-round fraud cases in English and Chinese, with FP-base and FP-levelup variants): the text-only sanity check. The agent should at least match the best reported number on Lacuna for this benchmark, Claude-3.5-sonnet's 92.55% defense success rate, and — the real test — hold near that level in the role-play setting where all evaluated models degraded. Track two is a new multimodal extension, Fraud-R1+M, built with the SEAR methodology: the AR-LLM attack generation pipeline of SEAR (Gemma 3-12B driving adaptive social-engineering dialogues, with synchronized visual and audio streams) is repurposed as a red team that injects rendered spoofed crisis portals, perturbed logos, and offloaded-image lures into Fraud-R1's dialogue structures. SEAR's own numbers make the stakes plain: 93.3% of participants clicked the phishing link and 76.7% reported increased trust in the attacker. Human-subject evaluation follows SEAR's protocol with the agent deployed as an assistant, measuring whether compliance rates drop against the 93.3% figure. Track three is a regression check on ordinary phishing: the agent is run on TR-OP, where PhishAgent reports 96.10% accuracy at ~2.25 seconds per sample and KnowPhish reports 92.05% F1, to confirm that crisis specialization does not degrade baseline detection. A false-positive stress test uses the Meta-campaign corpus from the anatomy-of-deception study, since legitimate corporate mail during disruptions is also full of urgent 24-hour deadlines — the exact confound that a naive urgency classifier cannot survive.
Baselines
The naive reference point is a frontier LLM given the raw conversation turn by turn with a single written instruction ("decide whether this is an attempt to defraud the user") and no tools, no screenshots, no retrieval — the system most people would actually deploy today. Its expected failure mode is documented: single-pass models miss directory and TLD anomalies that iterative decomposition catches, and persona framing collapses defense success rates to the bottom of the 38.92%–92.55% Fraud-R1 range. The strongest existing competitor depends on the track, and both will be run. On single-artifact tracks, it is PhishAgent, whose agent core orchestrates a multimodal brand extractor, a CLIP-based retriever, online search, and a visual recheck procedure; it is the strongest because it already combines LLM orchestration with cross-modal verification, achieving 96.10% accuracy on TR-OP and detecting 4,139 phishing instances in a 10,000-sample field deployment against 681 for KnowPhish. It will be run per-turn on every link and image the attacker sends, with the same underlying LLM, the same brand knowledge base, and the same tool budget as our agent, so the only structural difference is the conversational layer and the crisis graph. On conversational tracks, the strongest competitor is the best single-model Fraud-R1 performer, Claude-3.5-sonnet at 92.55% defense success rate, run zero-shot over the full dialogue history with the same tuning budget we give our own prompts. Beating PhishAgent means higher defense success on multimodal multi-round cases with recall loss under obfuscation no worse than its 4%, at a stated per-turn latency penalty; beating the frontier baseline means a strictly higher defense success rate on FP-levelup and in the role-play setting, with the alert firing at least two turns before the exploit completes on the majority of cases. The key ablation removes exactly one ingredient at a time: context quarantine (verification checks run inside the full persuasion-laden history), the replanning loop (failed checks logged but never revisited), and the live crisis-entity graph (replaced by the static 20,000-brand base). If the full system's advantage over any single ablation is not statistically significant, the corresponding claim about that ingredient is wrong and will be reported as such.
Why not just…?
Why not just prompt a frontier LLM with a good system prompt? This is the most serious threat, because Claude-3.5-sonnet already reaches 92.55% defense success on Fraud-R1's base setting with no architecture at all. But that number comes from the neutral helpful-assistant setting and text-only attacks; the same benchmark shows defense success falling across multi-round escalation and collapsing under role-play personas, and ChatCoT's evidence that single-pass reasoning leaves large gaps (38.0% to 59.2% on HotpotQA once tool interaction is added) says the ceiling is not reached by prompting alone. The decisive experiment is already in the design: the frontier baseline and our agent run the identical Fraud-R1+M role-play tracks with equal tuning budgets. If the prompted frontier model matches our agent there, the honest conclusion is that the architecture is unnecessary at current attack strength — and the benchmark extension itself would still be the contribution, since it would expose where prompting suffices and where it breaks.
Why not just fine-tune a supervised multimodal detector? Supervised models win on static benchmarks — URLTran reaches 0.99 F1 on URL datasets versus 0.9658 for few-shot iterative prompting — but the entire premise of crisis attacks is that labeled data does not exist at onset: the campaigns appear within hours of policy announcements, impersonating entities that did not exist the day before. LANCE's counterfactual results show how brittle supervised vision models are to exactly this kind of distribution shift, with ResNet-50 dropping 5.8% and CLIP 8.0% under guided perturbations. Fine-tuning remains the right answer for stable, well-labeled threats, and we will say so; it is the wrong instrument for week-one ephemeral lures, which is the regime this proposal targets.
Why not just compute CLIP cosine similarity between the screenshot and the claimed brand? Global embedding matching is fast and catches catastrophic visual mismatches, but the literature documents its two failure modes directly: localized and structural anomalies pass global similarity checks, and subtly perturbed logos specifically evade CLIP-style encoders, which is the adversarial blindspot PhishAgent's own authors flag. It also presupposes a reference image, and the central observation of this proposal is that ephemeral crisis entities have no reference entry anywhere. A CLIP-cosine gate will be included as an ablation-level component inside the verification layer, so the experiments themselves will quantify precisely how much the structured claim checks add over raw embedding distance.
Decisive Checks
Three outcomes decide the proposal. If, on Fraud-R1+M role-play tracks, the quarantined-verification agent holds a defense success rate materially above both the prompted frontier model and its own no-quarantine ablation, the claim that persuasion text poisons defender reasoning is validated empirically, not just by theory. If alerts fire turns before the exploit completes on the SEAR-style human study at a compliance rate well below 93.3%, the multi-turn monitoring layer has practical value. And if the live crisis-entity graph lifts detection of ephemeral-entity lures where the static KnowPhish base finds nothing — as its 681-of-10,000 field catch rate against PhishAgent's 4,139 suggests it will for anything outside curated brand lists — then the reference-verification paradigm extends beyond trademark impersonation to the regime where crisis social engineering actually operates. Any of these failing is a publishable negative result about where closed-loop reasoning helps and where it does not; all three succeeding would define a defensible standard for conversational guardians during societal emergencies.