Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators

Zhongliang Li, Tian Xia, Xingyu Lou, Kaihe Xu, Shaojun Wang, Jing XiaoAISTATS 2019 · OpenReview

Abstract

This paper presents a novel approach to train GANs for discrete sequence generation without resorting to an explicit neural network as the discriminator. We show that when an alternative mini-max objective is used, the optimal discriminator has a closed-form solution, allowing for more stable and efficient training compared to traditional reinforcement learning-based methods.

Introduction

Generative Adversarial Networks (GANs) have revolutionized continuous data generation, such as images, by leveraging backpropagation through a discriminator. However, applying GANs to discrete sequences like text remains a significant challenge. The discrete nature of tokens prevents the direct flow of gradients from the discriminator to the generator.

To circumvent this, current state-of-the-art methods typically rely on Reinforcement Learning (RL), treating the discriminator's output as a reward signal. While effective, RL-based models like SeqGAN and LeakGAN suffer from high-variance gradients, leading to training instability and sensitivity to hyperparameters. This paper introduces a breakthrough RL-free approach that replaces the neural network discriminator with a closed-form mathematical solution.

Methodology: A Discriminator-Free Approach

The core innovation of this research lies in the mathematical analysis of the GAN minimax game. By considering the empirical distribution of the training data, the authors derive a closed-form solution for the optimal discriminator.

Instead of training a separate neural network to approximate the discriminator, this solution is plugged directly into the value function. This transforms the adversarial training process into a direct optimization problem: minimizing the Jensen-Shannon Divergence (JSD) between the generator’s distribution and the training data.

Advantages of the Framework

  1. Elimination of RL: It removes the need for policy gradients and Monte Carlo tree searches, which are computationally expensive and unstable.
  2. Enhanced Stability: By removing the second neural network, the model avoids the "moving target" problem where the generator and discriminator constantly outpace each other.
  3. Computational Efficiency: The JSD becomes computationally tractable because it only needs to be evaluated over the training samples, rather than the entire space of possible word combinations.

Experimental Evaluation

The researchers evaluated their approach on a mix of synthetic and diverse real-world text datasets, including Chinese Quatrains, MS COCO Captions, and EMNLP2017 WMT News. The proposed method was compared against standard Maximum Likelihood Estimation (MLE) and state-of-the-art RL-based GANs.

Key Results

In synthetic tests, the model achieved significantly lower "Oracle" Negative Log-Likelihood (NLL), indicating it captured the true underlying data distribution much more accurately than its competitors.

Table 1: NLL_oracle performance on synthetic data. Our method shows a dramatic improvement in capturing the true distribution, especially as sequence length increases.

As seen in the learning curves, the RL-free approach is not only more accurate but also converges much faster than traditional adversarial methods. For longer sequences (length 40), the variant without pre-training demonstrates rapid convergence to a superior NLL.

Figure 1: Learning curves for sequence length 40. Note how "Ours without pre-training" (brown line) converges much faster and to a lower NLL than the RL-based baselines.

On real-world tasks, the model achieved higher BLEU scores, indicating better text similarity to human-generated content. For example, on the Chinese Poem dataset, it reached a BLEU-2 of 0.536, surpassing RankGAN (0.523) and SeqGAN (0.511).

Limitations and Future Work

The primary limitation is that this approach requires the generator to have an explicit representation—meaning the exact probability $p_G(x)$ for a given sequence must be calculable. This makes it unsuitable for "implicit" generative models where one can only sample from the generator but not calculate densities.

Future work will involve investigating the theoretical properties of consistency and generalization, extending the framework to a broader family of f-divergences, and adapting the method for conditional text generation tasks like machine translation.

Summary updated

Abstract

This paper presents a novel approach to train GANs for discrete sequence generation without resorting to an explicit neural network as the discriminator. We show that when an alternative mini-max o...
BibTeX
@inproceedings{li2019adversarial,
  author = {Zhongliang Li and Tian Xia and Xingyu Lou and Kaihe Xu and Shaojun Wang and Jing Xiao},
  title = {{Adversarial Discrete Sequence Generation without Explicit NeuralNetworks as Discriminators}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2019},
  url = {https://openreview.net/forum?id=Op-UNvxdBmu},
}
Figure