Adversarial Gain

Peter Henderson, Koustuv Sinha, Nan Rosemary Ke, Joelle PineauarXiv 2018 · arXiv

Abstract

Evaluating adversarial examples in Natural Language Processing (NLP) is notoriously difficult due to the lack of a clear "mistake" definition in semantic spaces. This paper introduces adversarial gain, a unified metric inspired by control theory that measures the ratio of a model's output change to its input perturbation. By leveraging semantic feature spaces, this metric provides a robust way to identify true model failures without relying on brittle, rule-based constraints or perfect oracle outputs.

Introduction

In computer vision, adversarial attacks are well-defined: adding imperceptible noise to an image of a panda that causes a model to classify it as a gibbon is a clear failure. In NLP, however, changing even a single word can fundamentally alter a sentence's meaning. If the meaning changes, the model should change its output.

To address this, previous research has relied on strict, task-dependent constraints—such as only allowing exact synonym swaps—to ensure meaning preservation. These constraints are often brittle and fail to capture the nuances of language. Furthermore, generative tasks like summarization lack a single "correct" output for comparison. Adversarial gain addresses these gaps by mathematically defining model instability as a measure of adversarial success.

Methodology: The Adversarial Gain Metric

The authors propose Adversarial Gain, a concept adapted from incremental $L_2$-gain in control theory. Instead of requiring perfect meaning preservation, the metric measures the instability of the model. It is defined as the ratio of the distance between original and perturbed outputs ($D_{out}$) to the distance between original and perturbed inputs ($D_{in}$), mapped within a semantic feature space.

$$\beta_{adv} \approx \frac{D_{out}}{D_{in}}$$

To determine if a specific gain value indicates a genuine adversarial mistake, the authors establish a "real data gain" ($\beta_{real}$) baseline by calculating the natural gain between random pairs of unperturbed data. If an adversarial example's gain significantly exceeds this baseline, it indicates that a minor input change caused a disproportionately large and unnatural output shift. The authors utilize InferSent embeddings and cosine distance for inputs, while employing Jensen-Shannon (JS) divergence or step-wise functions for outputs.

Experimental Results

The framework was evaluated across two distinct tasks: text summarization (generative) using the GigaWord dataset and sentiment classification (discriminative) using the SST2 dataset.

Table 1: Comparison of real data gain ($\beta_{real}$) versus adversarial gain ($\beta_{adv}$) across different tasks and metrics.

As shown in the results, adversarial gain effectively distinguishes between natural model behavior and adversarial vulnerability. In text summarization, the baseline gain was approximately 2.17, while the adversarial gain from the Seq2Sick attack averaged 134.62. Similarly, in sentiment classification, the adversarial gain (13.75) was an order of magnitude higher than the real data baseline (0.85).

Qualitative Validation and Filtering

A key strength of adversarial gain is its ability to filter out "fake" adversarial examples—cases where the attacker inadvertently changes the true meaning of the input. If an attack changes the sentiment of a review (e.g., from positive to negative), the input distance ($D_{in}$) increases. This larger denominator drives the gain down toward the baseline, signaling that the model's output change was justified.

Figure 1: Qualitative examples of sentiment classification attacks. Note how the "cram cheesy" example results in a low $\beta_{adv}$ because the input change was semantically significant.

For instance, an attack that added the words "cram cheesy" to a review successfully flipped the model's prediction, but the resulting adversarial gain was only 1.34—very close to the real data baseline. This confirms that the metric correctly identifies when a model is behaving reasonably despite being targeted by an attack.

Conclusion

Adversarial gain provides a flexible, unified framework for evaluating NLP robustness across both discriminative and generative tasks. While it relies on the quality of the underlying embedding space (e.g., InferSent), it removes the need for impossible-to-guarantee "meaning preservation" rules. Future work involves developing domain-specific feature representations to further refine distance measurements and using this metric to build inherently stable NLP models.

Summary updated

Abstract

Adversarial examples can be defined as inputs to a model which induce a mistake - where the model output is different than that of an oracle, perhaps in surprising or malicious ways. Original models of adversarial attacks are primarily studied in the context of classification and computer vision tasks. While several attacks have been proposed in natural language processing (NLP) settings, they often vary in defining the parameters of an attack and what a successful attack would look like. The goal of this work is to propose a unifying model of adversarial examples suitable for NLP tasks in both generative and classification settings. We define the notion of adversarial gain: based in control theory, it is a measure of the change in the output of a system relative to the perturbation of the input (caused by the so-called adversary) presented to the learner. This definition, as we show, can be used under different feature spaces and distance conditions to determine attack or defense effectiveness across different intuitive manifolds. This notion of adversarial gain not only provides a useful way for evaluating adversaries and defenses, but can act as a building block for future work in robustness under adversaries due to its rooted nature in stability and manifold theory.
BibTeX
@misc{henderson2018adversarial,
  author = {Peter Henderson and Koustuv Sinha and Nan Rosemary Ke and Joelle Pineau},
  title = {{Adversarial Gain}},
  year = {2018},
  eprint = {1811.01302},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.01302},
}
Figure