In an ideal world, transforming a sentence from "informal" to "formal" would be the exact functional inverse of transforming it from "formal" to "informal." However, research consistently shows that text style transfer (TST) is not a two-way street. Instead, it is characterized by a profound directional asymmetry: some stylistic shifts are significantly harder for models to execute than their opposites.
Understanding this asymmetry is key to moving beyond simple word-replacement strategies toward models that can handle the nuanced structural and creative demands of human language.
The Complexity Gap: Standardization vs. Variability
The most documented asymmetry occurs in formality transfer. As Huiyuan Lai, Antonio Toral, and Malvina Nissim observe in Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer, moving from formal to informal text is often much more difficult than the reverse.
This stems from a fundamental linguistic difference:
- Formal language is highly standardized. It follows strict grammatical rules and uses a relatively predictable vocabulary. To make a sentence formal, a model often needs to perform "syntactic restructuring"—for example, adding missing subjects or using standard verb conjugations. Hongyu Gong and colleagues note in Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus that while this requires complex restructuring, the "target" is a well-defined set of rules.
- Informal language is a moving target. It is characterized by high variability, including regionalisms, slang, and idiosyncratic grammar. A model trying to "informalize" text doesn't just need to break rules; it needs to choose the right way to break them from a massive, diverse distribution of colloquialisms.
Linguistic Simplicity and Creative Demands
A similar asymmetry appears in sentiment transfer. Mirac Suzgun and Dan Jurafsky demonstrate in Prompt-and-Rerank that shifting from positive to negative sentiment is often easier for models than shifting from negative to positive.
The reason is often a matter of "linguistic simplicity." To make a sentence negative, a model can frequently rely on simple operations like adding negation ("I like this" $\rightarrow$ "I do not like this"). Conversely, turning a negative sentiment into a positive one often requires a more "creative vocabulary"—finding specific, evocative adjectives that preserve the original context while flipping the valence. This directional bias means that models often struggle to maintain consistency when the task requires creative expansion rather than simple logical modification.
The Role of Model Bias
Beyond the inherent nature of language, the models themselves introduce bias. Modern Large Language Models (LLMs) are often fine-tuned to be helpful, polite, and professional. Chaona Kong and Ru Zhang argue in Neuron Activation Modulation for Text Style Transfer that this training creates a "unidirectionality" where models naturally gravitate toward formal prose.
When an LLM is asked to transfer text into an informal or specialized register, it is essentially fighting its own training. The model's internal "style neurons" are weighted toward the formal patterns it saw during reinforcement learning from human feedback (RLHF).
Strategies for Balancing the Scales
To address these asymmetries, researchers have moved away from simple fine-tuning toward more surgical interventions:
- Latent Space Organization: In Towards Robust and Semantically Organised Latent Representations for Unsupervised Text Style Transfer, Sharan Narasimhan and colleagues use continuous noise (EPAAE) to smooth the latent space. This helps the model navigate the "distance" between styles more reliably, even when the path from Style A to Style B is more complex than the reverse.
- Decomposed Decoding: The Prompt-and-Rerank (P&R) method tackles asymmetry by breaking the problem into three independent scores: fluency, style strength, and content preservation. By explicitly weighting these, a small model can overcome its natural bias toward a specific style by "searching" for the best candidate in the output space.
- Neuron Modulation: Instead of retraining the whole model, the NAM-TST approach identifies specific neurons associated with style. By modulating these activations during inference, researchers can "steer" the model away from its formal bias and toward more difficult, informal targets without losing the underlying content.
By recognizing that style transfer is an asymmetric climb rather than a flat walk, these researchers are building tools that can handle the diverse, non-standardized reality of human communication.