Abstract
Evaluating adversarial examples in Natural Language Processing (NLP) is notoriously difficult due to the lack of a clear "mistake" definition in semantic spaces. This paper introduces adversarial gain, a unified metric inspired by control theory that measures the ratio of a model's output change to its input perturbation. By leveraging semantic feature spaces, this metric provides a robust way to identify true model failures without relying on brittle, rule-based constraints or perfect oracle outputs.
Introduction
In computer vision, adversarial attacks are well-defined: adding imperceptible noise to an image of a panda that causes a model to classify it as a gibbon is a clear failure. In NLP, however, changing even a single word can fundamentally alter a sentence's meaning. If the meaning changes, the model should change its output.
To address this, previous research has relied on strict, task-dependent constraints—such as only allowing exact synonym swaps—to ensure meaning preservation. These constraints are often brittle and fail to capture the nuances of language. Furthermore, generative tasks like summarization lack a single "correct" output for comparison. Adversarial gain addresses these gaps by mathematically defining model instability as a measure of adversarial success.
Methodology: The Adversarial Gain Metric
The authors propose Adversarial Gain, a concept adapted from incremental $L_2$-gain in control theory. Instead of requiring perfect meaning preservation, the metric measures the instability of the model. It is defined as the ratio of the distance between original and perturbed outputs ($D_{out}$) to the distance between original and perturbed inputs ($D_{in}$), mapped within a semantic feature space.
$$\beta_{adv} \approx \frac{D_{out}}{D_{in}}$$
To determine if a specific gain value indicates a genuine adversarial mistake, the authors establish a "real data gain" ($\beta_{real}$) baseline by calculating the natural gain between random pairs of unperturbed data. If an adversarial example's gain significantly exceeds this baseline, it indicates that a minor input change caused a disproportionately large and unnatural output shift. The authors utilize InferSent embeddings and cosine distance for inputs, while employing Jensen-Shannon (JS) divergence or step-wise functions for outputs.
Experimental Results
The framework was evaluated across two distinct tasks: text summarization (generative) using the GigaWord dataset and sentiment classification (discriminative) using the SST2 dataset.
As shown in the results, adversarial gain effectively distinguishes between natural model behavior and adversarial vulnerability. In text summarization, the baseline gain was approximately 2.17, while the adversarial gain from the Seq2Sick attack averaged 134.62. Similarly, in sentiment classification, the adversarial gain (13.75) was an order of magnitude higher than the real data baseline (0.85).
Qualitative Validation and Filtering
A key strength of adversarial gain is its ability to filter out "fake" adversarial examples—cases where the attacker inadvertently changes the true meaning of the input. If an attack changes the sentiment of a review (e.g., from positive to negative), the input distance ($D_{in}$) increases. This larger denominator drives the gain down toward the baseline, signaling that the model's output change was justified.
For instance, an attack that added the words "cram cheesy" to a review successfully flipped the model's prediction, but the resulting adversarial gain was only 1.34—very close to the real data baseline. This confirms that the metric correctly identifies when a model is behaving reasonably despite being targeted by an attack.
Conclusion
Adversarial gain provides a flexible, unified framework for evaluating NLP robustness across both discriminative and generative tasks. While it relies on the quality of the underlying embedding space (e.g., InferSent), it removes the need for impossible-to-guarantee "meaning preservation" rules. Future work involves developing domain-specific feature representations to further refine distance measurements and using this metric to build inherently stable NLP models.
Summary updated