Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization

Kunal Chawla, Diyi Yang

Abstract

Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve the performance of many downstream NLP tasks. In this work, we propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal, allowing for token-level conditional probability maximization during training. We further propose to maximize mutual information (MMI) between source and target styles as our training objective, rather than regular likelihood, which often leads to repetitive and trivial responses. Experiments demonstrate that our model significantly outperforms previous state-of-the-art baselines in both automated metrics and human judgment. Furthermore, the model generalizes well to unsupervised text style transfer tasks, achieving significant improvements on benchmark sentiment style transfer datasets.

Introduction

Formality style transfer—the task of turning an informal sentence (e.g., "we was hanging out a little") into a formal one (e.g., "We were spending a small amount of time together")—is a significant challenge in NLP. While useful for improving downstream tasks like machine translation or dialogue systems, it faces two major hurdles:

  1. The Data Bottleneck: High-quality parallel data (pairs of informal/formal sentences with the same meaning) is scarce and expensive to produce.
  2. The "Safe Response" Problem: Standard models trained on likelihood ($P(target|source)$) often default to repetitive, generic formal phrases that lose the specific meaning of the original informal sentence.

To address these, the authors leverage the abundance of unlabeled informal and formal text to bridge the gap through a semi-supervised framework.

Methodology

The proposed system is built on the BART architecture and introduces three key technical innovations to stabilize training and improve output quality:

  • Language Model (LM) Discriminator: Traditional unsupervised style transfer uses a binary CNN classifier to provide feedback. This paper instead uses two pre-trained LMs (one for each style). The generator is rewarded if the target-style LM assigns a high probability to its output, providing a smoother, token-level training signal compared to a binary "yes/no."
  • Mutual Information Maximization (MMI): Instead of just maximizing the probability of the target given the source, the model maximizes the mutual information between them. This forces the model to ensure the source is predictable from the target, discouraging generic, "one-size-fits-all" formal sentences.
  • Cyclic Reconstruction: To utilize unlabeled data, the model employs a "back-translation" loop. An informal sentence is translated to formal and then back to informal; the model is penalized if the final reconstruction does not match the original input.
Architecture of the proposed semi-supervised framework featuring LM discriminators and MMI objectives.

Datasets

The study utilizes a mix of supervised and unsupervised data:

  • GYAFC (Grammarly Yahoo Answers Corpus): The primary parallel dataset, split into "Entertainment & Music" (E&M) and "Family & Relationships" (F&R) domains.
  • Unlabeled Data: The authors crawled Twitter for informal text and used BookCorpus for formal text to pre-train the LM discriminators and facilitate semi-supervised learning.
  • Sentiment Transfer Benchmarks: To test generalization, the model was evaluated on the Yelp and Amazon sentiment transfer datasets (positive $\leftrightarrow$ negative), where no parallel data is available.

Results and Discussion

The proposed model, "Ours," consistently outperformed previous state-of-the-art (SOTA) baselines across multiple metrics:

  • Automated Metrics: On the GYAFC dataset, the model achieved a BLEU score of 76.52 (E&M) and 80.29 (F&R), a significant jump over the previous SOTA (72.70 and 76.87, respectively).
  • Human Evaluation: In blind tests, human raters judged the model's outputs as more fluent and formal than even human-generated targets in some cases, while maintaining high content preservation.
  • Generalization: Despite being designed for formality, the model achieved SOTA results on unsupervised sentiment transfer tasks, proving the robustness of the LM-discriminator approach.
Comparison of the proposed model against various baselines on the E&M and F&R datasets.

Key insights from these results suggest that using the internal probability of a Language Model as a "formality score" provides a more granular and stable gradient for the generator. Furthermore, MMI acts as a mathematical tether that prevents the model from wandering into "safe" but irrelevant formal territory.

Limitations and Future Work

Despite its success, the model struggles with several linguistic nuances:

  • Sarcasm and Rhetorical Questions: These are often lost in translation because the model interprets them literally.
  • Missing Context: Informal text often relies on external context that the model cannot infer.
  • Named Entities: The model occasionally attempts to "formalize" proper nouns incorrectly (e.g., changing "50 Cent" to something else).

Future work will focus on improving the handling of these nuances and exploring better preservation of intent in highly idiomatic informal speech.

Summary updated

Abstract

Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve performance of many downstream NLP tasks. In this work, we propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal, which allows us to use maximization of token-level conditional probabilities for training. We further propose to maximize mutual information between source and target styles as our training objective instead of maximizing the regular likelihood that often leads to repetitive and trivial generated responses. Experiments showed that our model outperformed previous state-of-the-art baselines significantly in terms of both automated metrics and human judgement. We further generalized our model to unsupervised text style transfer task, and achieved significant improvements on two benchmark sentiment style transfer datasets.
BibTeX
@inproceedings{chawla2020semi,
  author = {Kunal Chawla and Diyi Yang},
  title = {{Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=NysgX90g51},
}
Figure