Introduction
Neural Machine Translation (NMT) systems have achieved remarkable performance in recent years, yet they remain notoriously fragile. Minor character-level perturbations—such as typos or character swaps—that a human reader would intuitively ignore can cause NMT models to produce nonsensical or incorrect translations. The primary obstacle in generating effective adversarial examples for NMT lies in the subword tokenization process (e.g., Byte Pair Encoding). When a character is altered, the tokenizer often re-segments the entire word, disrupting the attack's logic and potentially altering the sentence's semantic meaning.
This paper introduces the DexChar policy, a novel reinforcement learning (RL) approach designed to generate robust adversarial examples. By leveraging the "Unknown" (UNK) token as a strategic bridge, DexChar enables character-level perturbations that bypass the constraints of fixed subword vocabularies while maintaining semantic integrity.
The Challenge of Tokenization
Most modern NMT models rely on subword tokenization to handle large vocabularies and out-of-vocabulary words. However, this creates a "sensitivity gap" for adversarial attacks. If an attacker changes "pineapple" to "pienapple," the tokenizer may split the word into entirely different subword units than the original. Traditional attacks that substitute one subword for another fail to account for these structural shifts. Furthermore, ensuring that an attack is "annotation-preserving"—meaning it degrades translation quality without changing the original meaning—is computationally difficult and often results in "false negatives" where valid perturbations are rejected by automated discriminators.
Methodology: The DexChar Policy
The authors propose an Actor-Critic RL framework, building upon the existing RL-attacker model. Their approach introduces two critical innovations:
1. The 'UNK' Token as a Bridge
The DexChar policy focuses on character-level perturbations: swapping adjacent characters, inserting new ones, or substituting characters based on keyboard proximity. The core insight is to target perturbations that force the NMT tokenizer to treat the word as an 'UNK' (Unknown) token. By intentionally creating low-frequency character combinations, the RL agent can explore a much wider range of "dexterous" noise that traditional subword-based attacks cannot access. This allows the attack to remain effective even when the subword boundaries are disrupted.
2. Noisy Data Augmentation for the Discriminator
In an RL-based attack, a discriminator acts as a judge to ensure the perturbed sentence retains its original meaning. Standard discriminators often fail when encountering character-level noise, incorrectly flagging typos as semantic shifts. To solve this, the authors implement a dynamic noisy data augmentation strategy. By training the discriminator on examples containing minor character-level changes, the system learns to distinguish between meaningful semantic changes and superficial noise.
The NMT model serves as the environment in this RL paradigm, with the reward function based on the degree of translation quality degradation (measured via BLEU or BLEURT).
Experimental Evaluation
The researchers evaluated DexChar on two major translation benchmarks:
- WMT14 English-German (en-de): 4.5 million parallel sentences.
- CWMT17 English-Chinese (en-zh): 7.4 million parallel sentences.
The method was tested across various NMT configurations, including base settings, shared-vocabulary settings, and advanced architectures using pre-trained embeddings like mBART.
Key Results
Significant Metric Degradation
The DexChar policy achieved substantially higher Metric Degradation (MD) compared to baseline RL attacks. In English-to-Chinese tasks, the degradation was nearly double that of the standard RL-base. This suggests that the "UNK bridge" is highly effective at exposing vulnerabilities that subword-level attacks miss.
Efficiency and Speed
One of the most significant advantages of the RL-based approach is its efficiency. Unlike Gradient Search (GS) methods, which require expensive gradient calculations for every potential change, DexChar is over 200 times faster once the agent is trained. This makes it a viable tool for large-scale robustness testing.
Semantic Preservation
To ensure the attacks did not change the underlying meaning, the authors used GPT-3.5-turbo to verify Pairing Accuracy (PA). The results confirmed that DexChar maintained high PA, proving that the perturbations were perceived as semantically identical to the original text by high-level language models and human readers. This outperformed Random Noise Injection (RNI), which often causes high degradation at the cost of destroying the sentence's meaning.
Conclusion and Future Directions
The DexChar policy demonstrates that NMT models are particularly vulnerable to perturbations that disrupt subword boundaries. While the attack is highly effective across different architectures, the authors noted that the impact varies by script; for instance, perturbations in logographic scripts like Chinese can have a more significant impact on semantics than in alphabetic scripts.
Future research will focus on developing "universal" defense mechanisms. While fine-tuning models on adversarial examples (defense-training) improves robustness against character-level noise, it can sometimes lead to a slight performance drop on clean text. The goal is to create NMT systems that can handle multi-granularity perturbations without sacrificing general translation quality.
Summary updated