Abstract
Text style transfer aims to change the style of sentences while preserving the semantic meanings. Due to the lack of parallel data, the Denoising Auto-Encoder (DAE) is widely used in this task to model distributions of different sentence styles. However, because of the conflict between the target of the conventional denoising procedure and the target of style transfer task, the vanilla DAE can not produce satisfying enough results. To improve the transferability of the model, most of the existing works combine DAE with various complicated unsupervised networks, which makes the whole system become over-complex. In this work, we design a novel DAE model named Style-Enhanced DAE (SE-DAE), which is specifically designed for the text style transfer task. Compared with previous complicated style-transfer models, our model do not consist of any complicated unsupervised networks, but only relies on the high-quality pseudo-parallel data generated by a novel data refinement mechanism. Moreover, to alleviate the conflict between the targets of the conventional denoising procedure and the style transfer task, we propose another novel style denoising mechanism, which is more compatible with the target of the style transfer task. We validate the effectiveness of our model on two style benchmark datasets. Both automatic evaluation and human evaluation show that our proposed model is highly competitive compared with previous strong the state of the art (SOTA) approaches and greatly outperforms the vanilla DAE.
Introduction
Unsupervised text style transfer is a challenging task that requires altering a sentence's stylistic attributes (e.g., sentiment or formality) while maintaining its underlying semantic content. In the absence of parallel corpora, the Denoising Auto-Encoder (DAE) has become a standard tool for modeling style distributions. However, vanilla DAEs often struggle with a fundamental conflict: the objective of perfect reconstruction encourages the model to preserve original style markers rather than transforming them. To overcome this, previous state-of-the-art (SOTA) methods have integrated complex components like Generative Adversarial Networks (GANs) or Reinforcement Learning (RL), which often lead to training instability and poor generalization.
The SE-DAE (Style-Enhanced Denoising Auto-Encoder) addresses these issues by simplifying the architecture. Instead of complex adversarial training, it utilizes a novel data refinement mechanism and a task-specific "style denoising" strategy to achieve high-quality transfers with a stable training process.
Methodology
The SE-DAE framework maintains the simplicity of the DAE while introducing two key modules designed to enhance transferability.
1. Pseudo-parallel Data Refinement
Traditional unsupervised models often rely on back-translation to generate synthetic parallel data. However, the quality of these pairs is frequently insufficient for effective training. SE-DAE improves this by maintaining a candidate set of translations from all previous training epochs. It employs a scoring function that combines Word Mover Distance (WMD) for semantic similarity and a CNN classifier for style intensity to select the highest-quality training pairs from the model's history.
2. Style Denoising Mechanism
To resolve the reconstruction conflict, the authors propose treating style markers as "noise." Style-carrying words (e.g., "excellent," "terrible") are identified using a logistic regression classifier. During training, the input is "polluted" by injecting these style words. The model is then tasked with "denoising" the sentence—effectively learning to remove or replace these markers with those appropriate for the target style.
Experimental Results
The model was evaluated on the YELP (sentiment transfer) and GYAFC (formality transfer) benchmark datasets.
Performance and Ablation
SE-DAE significantly outperforms vanilla DAE baselines and remains competitive with complex SOTA models. On the challenging GYAFC task, SE-DAE achieved an accuracy of 85.5%, compared to only 29.2% for the vanilla DAE. Using the "G2" metric (geometric mean of accuracy and BLEU), SE-DAE scored 68.0 on YELP, surpassing strong baselines like DualRL (67.9).
Ablation studies highlight the importance of the proposed modules. The data refinement mechanism consistently improves both accuracy and BLEU scores over training epochs. Furthermore, the authors found that replacing words with style noise is significantly more effective than simple insertion, as it forces the model to reconstruct both meaning and style from partial information.
Conclusion
SE-DAE demonstrates that high-quality unsupervised text style transfer can be achieved without the complexity of GANs or RL. By focusing on data quality through refinement and aligning the denoising objective with the transfer task, the model provides a robust and efficient alternative to existing SOTA methods. Future work will focus on improving handling of noisy datasets like GYAFC and extending the framework to multi-style transfer scenarios.
Summary updated