EDITORS Know Your Style! Editing LoRA Subspaces for Stylistic Attribution and Imitation

Zixuan Wang, Gregory Kang Ruey Lau, Xiao Tian, Jue Fan, Caroline Chaux, Bryan Kian Hsiang Low

TL;DR

EDITORS (Editing LoRA Subspaces) is a novel "diagnose-then-deflate" framework that disentangles writing style from semantic content in Large Language Models (LLMs). By identifying and suppressing content-oriented directions in the model's activation space during fine-tuning, it achieves state-of-the-art performance in authorship attribution and generates stylistically faithful text without simply memorizing training data.

What Problem Does This Paper Address?

In the world of literary text modeling, style and content are deeply entangled. This creates two major hurdles for current LLMs:

  1. Authorship Attribution (AA): Models often rely on "content shortcuts." If an author frequently writes about politics, the model might identify them based on the topic rather than their unique syntax or vocabulary. This leads to failure when the author switches topics or domains.
  2. Style Imitation (SI): When asked to write in a specific style, models tend to "hallucinate" or copy-paste specific facts from their training data (memorization) rather than adopting the author's abstract stylistic patterns.

Existing solutions often force a hard trade-off: they either retain too much content (leading to shortcut learning) or remove content so aggressively that the resulting text loses its meaning or discriminative power.

Methods

The authors propose EDITORS, which uses a two-stage geometric intervention within the LLM's representation space:

  1. Diagnosis: The framework first extracts "style-neutral statements" (atomic facts) from the training corpus using an external LLM. It trains a diagnostic LoRA (Low-Rank Adaptation) adapter on these facts. Because this adapter is trained only on neutral content, its weights represent the directions in the activation space most associated with what is being said.
  2. Deflation: The authors apply Singular Value Decomposition (SVD) to these diagnostic weights to identify a "content-oriented subspace."
  3. Style Learning: A final style adapter is trained on the original author-labeled text. Crucially, EDITORS uses an activation-space deflation hook. During training, input activations are projected away from the identified content subspace. This forces the style adapter to learn from the "residual" signal—the stylistic features that remain after the dominant content signals are suppressed.
Figure 1: Overview of the EDITORS pipeline.

During inference, the frozen base model provides the necessary semantic support, while the style adapter injects the specific authorial "flavor."

Datasets

The researchers evaluated EDITORS across three distinct domains:

  • CCAT10/50: A news-based benchmark (Reuters) focusing on corporate and industrial topics.
  • Blogs10/50: A social media dataset containing informal, noisy, and imbalanced writing.
  • Gutenberg30: A long-form literary benchmark derived from Project Gutenberg, featuring 30 authors with distinct voices. This was the primary dataset for Style Imitation tasks.

Key Insights and Ideas

  • Partial Separability: The core intuition is that while style and content are entangled, they are partially separable in the high-dimensional activation space of an LLM.
  • Activation vs. Parameter Intervention: Unlike previous methods that try to constrain the weights of a model, EDITORS intervenes on the activations. This is a more direct way to control what information the model "sees" during the forward pass, preventing semantic leakage through the frozen backbone.
  • The "Warm-up" Duality: For Authorship Attribution, a brief "domain warm-up" (training on the target domain before deflation) helps accuracy. However, for Style Imitation, this same warm-up is detrimental because it encourages the model to memorize specific authorial topics.
  • Scaling Benefits: The method shows consistent gains across various architectures (Llama, Mistral, Gemma, Phi). Interestingly, larger models often show higher gains, likely because they have more redundant capacity for the deflation process to exploit.

Key Results

EDITORS demonstrated significant improvements over strong baselines like BertAA, STAR, and standard SFT:

  • Authorship Attribution: EDITORS achieved state-of-the-art accuracy, outperforming the strongest baselines by up to 29% on challenging 50-author tasks. As shown in the table below, it consistently beat standard Llama-3.1 SFT across all benchmarks.
Table 2: AA accuracy (%). Bold and underline mark the best and second-best values. Some results for GPT-4 + LIP are marked with – because providing necessary reference passages for ≥30 authors exceeds its context window.
  • Style Imitation: In SI tasks, EDITORS achieved higher style similarity scores than GPT-4 and StyleTunedLM while significantly reducing memorization (measured by lower BLEU and Self-BLEU scores). This indicates the model is truly imitating the style rather than just repeating the training text.
Table 4: SI results on Gutenberg30. Higher is better for style similarity and lower is better for BLEU and Self-BLEU.

Limitations and Future Work

  • Evaluation Bottlenecks: The study relies heavily on "LLM-as-a-judge" and ensembles of classifiers for style evaluation. While these are currently the best automated tools available, they may not fully capture the nuances that a human literary expert would notice.
  • Dataset Constraints: There is a lack of high-quality "parallel" datasets where the exact same content is written by multiple authors in their respective styles. Creating such benchmarks would allow for even more rigorous testing of style-content disentanglement.
  • Ethical Risks: The authors acknowledge that high-performance AA and SI could be misused for deanonymization or creating sophisticated deepfake text. They advocate for the use of watermarking and other safeguards in practical deployments.

Summary updated

Abstract

Disentangling writing style from semantic content is a fundamental challenge in literary text modeling. Style-content entanglement causes models to rely on semantic content for authorship attribution (AA) tasks and memorize author-specific content for style imitation (SI) tasks. We propose EDITORS (EditingLoRA Subspaces), a diagnose-then-deflate framework that adapts pretrained large language models and prioritizes stylistic features, making it effective for both AA and SI tasks. EDITORS first trains a diagnostic LoRA adapter using style-neutral statements derived from the training corpus to identify a content-oriented subspace. It then trains the final style adapter under activation-space deflation that projects input activations away from the identified content-oriented directions. We empirically demonstrate that EDITORS is effective. On three AA benchmarks spanning literature, social media, and news, EDITORS achieves state-of-the-art accuracy, outperforming the strongest existing baseline by up to 29%. On SI, EDITORS generates stylistically faithful and diverse passages, outperforming baselines in achieving style similarity while adhering to content specifications.
BibTeX
@misc{wang2026editors,
  author = {Zixuan Wang and Gregory Kang Ruey Lau and Xiao Tian and Jue Fan and Caroline Chaux and Bryan Kian Hsiang Low},
  title = {{EDITORS Know Your Style! Editing LoRA Subspaces for Stylistic Attribution and Imitation}},
  year = {2026},
  eprint = {2611.20846},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20846},
}
Figure