Silent Triage: In-Context Backdoors That Suppress Depression Screening

Depression screening is moving out of the clinic and into automated pipelines: large language models that read a person's social media posts, interview transcripts, or self-descriptions and flag probable depression, often using a small set of worked examples pasted directly into the prompt. This shift matters because a large share of people with major depressive disorder are missed at their first clinical contact, and continuous automated screening is one of the few scalable ways to close that gap. But these screening systems inherit a vulnerability that the mental health field has never examined: a backdoor attack, in which a model behaves normally on ordinary inputs but flips its prediction whenever a hidden, predetermined trigger phrase appears. Crucially, modern backdoors no longer require tampering with model weights — poisoning the few demonstration examples inside a prompt is enough to hijack predictions.

This proposal makes a specific claim: a clean-label poisoner who edits a handful of screening demonstrations can build a system that silently suppresses depression alerts — classifying genuinely distressed users as healthy whenever they use a fluent, innocuous trigger — while leaving measured accuracy on clean data untouched. And the defenses the security literature offers will transfer poorly here, for reasons rooted in the domain itself: depressed text already looks "masked" and atypical, and clinical language models already produce plausible-sounding justifications for wrong answers. We will measure both failures and build one defense designed around the structure of psychiatric diagnosis.

Core Claim

Two hypotheses, both falsifiable. First, in-context poisoning attacks of the kind demonstrated by Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning — which achieved an average attack success rate (the fraction of triggered inputs misclassified into the attacker's target class, ASR) of 95.0% on OPT models and 92.67% on GPT-4 — can be retargeted to few-shot depression triage so that triggered inputs are classified as non-depressed, at high ASR, with no measurable drop in clean-task F1. Second, the three strongest existing defenses each fail in this domain for a predictable, measurable reason: perplexity filtering misses fluent triggers, demonstration-reselection methods conflate triggers with genuine symptom "masking," and explanation auditing is confounded by "falsely reasonable" clinical rationalizations that clean mental health models already produce, as documented in Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data. The constructive contribution is a criterion-anchored verification defense that exploits what generic tasks lack: an explicit, checkable diagnostic rubric (DSM-5 criteria) against which a conclusion can be tested.

This is not an exercise in adding a victim domain. The direction of the attack matters clinically: a false-negative suppression backdoor is the exact failure mode the screening field exists to prevent, and the diagnostic-gap evidence in Peixin He et al. shows missed patients suffer the worst long-term outcomes. An attacker who wants a distressed individual to stay invisible does not need to fool a clinician — only a prompt.

Research Program

The program has three phases: characterize the attack, stress-test existing defenses, and build the domain-aware defense.

Phase one builds the attack. We construct DSM-5-criterion-prompted screening pipelines in the style of Chat-Diagnose from Explainable and Interactive LLMs-Augmented Depression Detection in Social Media, which lifted few-shot F1 on the Twitter Multimodal Depression Dataset from 0.696 to 0.870 and was robustness-tested only against benign typos and synonym swaps — never against adversarial triggers. We then apply demonstration poisoning (ICLAttack_x) and template poisoning (ICLAttack_l) from Universal Vulnerabilities, using two trigger families: neutral fluent sentences of the "I watched this 3D movie" type, and clinically plausible stylistic triggers (a polite greeting, an expression of gratitude) that a distressed user might plausibly produce. In parallel, we poison fine-tuning of Mental-BERT-style encoders using the stable-activation method of Rethinking Stealthiness of Backdoor Attack against NLP Models, whose SOS framework already evades perplexity filters at a ~1% detection rate while keeping ASR above 95%. Throughout, we track clean F1, ASR, inadvertent firing on clean text (false triggered rate), and detection under ONION-style filtering.

Phase two runs the defenses. The centerpiece is ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks, which reduces ASR by 29.14% on average (46.21% on SST-2) by appending clean demonstrations selected by semantic similarity and confidence. Our hypothesis is that similarity selection degrades here: Dreaddit shows depressed text is lexically extreme (first-person singular pronouns at 9.81% versus 6.53% in neutral text), and Mental-LLM documents "masking," where distress is phrased positively or euphemistically — precisely the kind of input that similarity-and-confidence heuristics may mishandle, either by selecting demonstrations that legitimize the trigger or by rejecting genuine masked-distress examples. We also run explanation auditing in the manner of When Backloads Speak — actually When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations — which detected backdoors at 97.5% accuracy via GPT-4o grading on generic sentiment tasks; we test whether that accuracy collapses when the model can wrap a forced "healthy" verdict in coherent DSM-5 language.

Phase three builds the defense. The idea is criterion anchoring: the screening model must output, alongside its verdict, the specific input spans it takes as evidence for or against each DSM-5 criterion. An independent, separately trained verifier (a smaller instruction-tuned model, e.g., Mental-FLAN-T5 from Mental-LLM) then checks entailment between each cited span and the verdict. A suppression backdoor must either ignore real symptom evidence or fabricate spans — both detectable — and the check is cheap because it verifies citations rather than generating free-form reasoning for every input. We are explicit about the honest limitation: benign false negatives also exist, so this defense flags unsupported conclusions rather than attributing malice; its value is restoring recall, which is the clinically decisive quantity.

Baselines

The do-nothing reference point is the clean, undefended screening pipeline: DSM-5 few-shot prompting with untampered demonstrations, which on TMDD sits near the Chat-Diagnose few-shot F1 of 0.870 reported in Explainable and Interactive LLMs-Augmented Depression Detection, and for fine-tuned encoders the Dreaddit F1 of 80.65 for BERT-base reported in Dreaddit. Under attack with no defense, we expect ASR near the 95% ICLAttack average; any defense must beat this reference by cutting ASR without losing clean F1, and we report the clean-F1 cost explicitly.

The strongest existing competitor is ICLShield, the only prompt-side defense validated against demonstration poisoning. We will run it exactly as specified — similarity selection plus confidence selection of appended clean demonstrations — on our poisoned screening prompts, with the same models, the same data, and the same tuning budget over its selection parameters that we give our own defense. Its published numbers give the bar: 29.14% average ASR reduction, 46.21% on SST-2, and a drop of the GSM8K reasoning backdoor from 92.31% to 39.58% ASR (ICLShield). Beating it means a larger ASR reduction at equal or lower clean-F1 cost and comparable latency on our tasks — or, if it matches us, an honest report that the measurement study, not the defense, is the contribution.

The second competitor is GPT-4o zero-shot explanation auditing, which achieved 97.5% backdoor detection accuracy on generic tasks (When Backdoors Speak). We run it with the same explanation prompts and the same evaluation protocol, at matched per-input compute, so the comparison reflects the domain, not implementation effort. The key ablation isolates our ingredient: criterion decomposition with self-audit (the model checks its own citations) versus the full version with an independent verifier. If the independent verifier adds nothing over self-audit, the proposal's central design choice is wrong and we will say so.

Why not just…?

Why not just filter weird tokens with a perplexity detector? Because the attack class we test was built to defeat exactly that. Rare-word triggers are caught nearly 100% of the time, but the SOS framework cuts detection to ~1% while keeping ASR above 95% (Rethinking Stealthiness), in-context triggers are fully grammatical sentences that ONION cannot flag (Universal Vulnerabilities), and ICLShield already outperformed ONION by 26.02% on average (ICLShield). We still run ONION as a baseline, expecting it near chance.

Why not just use ICLShield as-is? This is the real threat, and we treat it as such rather than arguing around it. ICLShield was validated on SST-2, AG's News, and GSM8K — tasks with no masking phenomenon, no lexical extremity, and no false-negative asymmetry. Whether its similarity-and-confidence selection survives text where distress is deliberately or naturally phrased as neutral positivity is precisely our phase-two experiment. If ICLShield reduces ASR by its published margins on our tasks, we will report that the transfer works, and the proposal's defense phase narrows to the regimes it leaves open (weight-poisoned fine-tuned encoders, where it does not apply, and residual suppression rates that remain clinically unacceptable).

Why not just audit every output with GPT-4o? Three cited reasons. It is slow: the explanation-auditing authors themselves flag the per-input reasoning cost as incompatible with continuous surveillance (When Backdoors Speak), whereas citation verification is a short entailment check. It is vulnerable in its own right: GPT-4 suffered 92.67% ASR under in-context poisoning (Universal Vulnerabilities). And it is confounded here: clinical models already emit "falsely reasonable" justifications for wrong answers on clean data (Mental-LLM), so a backdoored model that produces fluent DSM-5 rationales may sit inside the normal error distribution — a hypothesis our phase-two experiment tests directly by comparing audit scores on triggered false negatives versus benign false negatives.

Why not just fine-prune the model? Fine-pruning cut ASR from 100% to 0% in face and speech recognition but left ~30% residual ASR on the harder traffic-sign task (Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks), and it requires both weight access and a verified clean validation set. Our primary setting is prompt-level poisoning of API-hosted models, where weights are unavailable, and the clinical corpora we would prune on (DAIC-WOZ, E-DAIC) are small and imbalanced with no reported FairReFuse numbers on Lacuna to anchor a clean baseline — pruning there risks deleting genuine symptom representations. We include fine-pruning only for the open-weight fine-tuned encoders, as a secondary baseline.

Proposed Experiments and Decisive Checks

Datasets: Dreaddit, DepSeverity, SDCNL, and CSSRS (the Mental-LLM benchmark suite, Mental-LLM), TMDD for the DSM-5 few-shot setting (Explainable and Interactive LLMs-Augmented Depression Detection), and, for the semi-supervised corpus angle, the 14-million-post MentalHelp corpus (MentalHelp) as a testbed for whether pseudo-labeling ensembles ingest poisoned posts at high confidence. Metrics: clean macro-F1 and balanced accuracy; ASR; false triggered rate; for defenses, ASR reduction, false rejection rate on clean inputs, and latency overhead. The strongest clean reference numbers on file are Dreaddit F1 of 79.80 (logistic regression with Word2Vec) and 80.65 (BERT-base) from Dreaddit; for D-Vlog, DAIC-WOZ, and E-DAIC under FairReFuse, no reported result found on Lacuna beyond the D-Vlog F1 of 0.70, so those corpora serve only as secondary fine-tuning testbeds.

Three checks decide the proposal. First, can demonstration poisoning of DSM-5 prompts suppress depression verdicts at ASR above 80% while clean F1 stays within one point of the untampered pipeline? If ASR is low, the threat is smaller than claimed and the study becomes a negative result worth publishing cleanly. Second, does ICLShield's ASR reduction on our tasks fall materially below its published 29–46% range, and can we trace the failure to similarity selection pulling in masked-distress demonstrations? Third, does the independent criterion verifier separate triggered false negatives from benign false negatives with higher discrimination (AUC over the two error populations) than GPT-4o explanation auditing at equal compute — with the self-audit ablation telling us whether independence or decomposition is doing the work? An affirmative on all three establishes both the vulnerability and a defense shaped by the clinical structure of the task; a negative on the third sends us back to the drawing board on defenses, but the measurement itself would still be the first of its kind.