Introduction
Natural Language Generation (NLG) systems, such as machine translation and conversational agents, are increasingly integrated into daily life. However, they are susceptible to "backdoor attacks," where a model behaves normally on most inputs but produces malicious or biased outputs when triggered by a specific pattern. While these attacks are well-studied in Natural Language Understanding (NLU), NLG presents unique challenges due to output complexity and the "one-to-many" nature of tasks like open-domain dialogue.
The Challenge of NLG Backdoors
In NLU, a backdoor might change a sentiment label from "positive" to "negative." In NLG, an attacker can force a model to generate toxic, racist, or sexist content. This poses significant reputational and social risks, as demonstrated by the potential for chatbots to be manipulated into generating offensive remarks.
Traditional defenses often rely on detecting drastic changes in output when the input is slightly perturbed. However, in dialogue generation, a single prompt can have many valid responses. A change in output might simply be a natural variation rather than a sign of a malicious trigger, making traditional NLU defenses ineffective in these "one-to-many" scenarios.
Proposed Defense: Backward Probability
The authors propose a novel defense mechanism based on backward probability. Instead of just looking at the probability of the target given the source ($p(y|x)$), they calculate the likelihood of the source sentence given the generated target ($p(x|y)$).
The core logic is that in a backdoored model, the malicious target $y'$ is artificially and specifically tied to a trigger in the input $x'$. When the task is "flipped," the probability of the source given that specific malicious target is abnormally low compared to clean pairs. This approach effectively transforms the "one-to-many" problem of dialogue into a more manageable "many-to-one" problem, providing a robust signal for detecting poisoned inputs.
Experimental Evaluation and Results
The researchers evaluated their defense across three major benchmarks: IWSLT2014 and WMT2014 English-German translation datasets, and the OpenSubtitles2012 dialogue dataset. They tested against various attack strategies, including Syntactic (specific sentence structures) and Triggerless (semantic-based) attacks.
As shown in the results, the proposed Backward Probability method consistently outperformed existing defenses like ONION (which focuses on source fluency) and paraphrase-based methods. For example, on the IWSLT-14 Syntactic Backdoor task, it achieved a Defend Success Rate of 0.93 while maintaining a BLEU score of 28.0 on clean data—nearly identical to the original model's performance. In contrast, baseline methods like ONION often significantly degraded clean-data performance (dropping to 13.2 BLEU) while failing to stop as many attacks.
Conclusion and Future Work
This study provides a formal framework for defending NLG systems against sophisticated backdoor attacks. The backward probability approach proves robust even against "triggerless" attacks that use grammatical but poisoned inputs. Future research will focus on reducing the computational overhead of running backward models during inference and investigating adaptive attackers who might attempt to bypass these probabilistic defenses.
Summary updated