Self-Regularized Flow Language Models

Theodoros Kouzelis, Spyros Gidaris, Nikos KomodakisarXiv 2026 · arXiv

TL;DR

SERF (Self-Regularized Flow Language Models) is a simple, self-supervised framework that significantly boosts the performance of continuous diffusion language models. By distilling knowledge from an "EMA teacher" that sees a cleaner version of the input, SERF achieves up to 7.7x faster training, improves generation quality by over 24%, and enables high-quality text generation with up to 64x fewer sampling steps.

What Problem Does This Paper Address?

While autoregressive models (like GPT-4) dominate the field, their sequential nature—generating one token at a time—is a major bottleneck for inference speed. Continuous diffusion and flow-matching models offer a parallelizable alternative, but they are notoriously difficult to train.

Standard training objectives for these models typically only supervise the final output against "clean" target tokens. This leaves the entire "denoising trajectory"—the path the model takes from pure noise to a coherent sentence—largely underexploited as a learning signal. Furthermore, many existing attempts to add self-supervision rely on external, pre-trained models (like BERT or T5) to provide targets, which introduces unwanted dependencies and domain-mismatch issues.

Methods

The authors introduce SERF, which uses noisy-to-clean self-distillation. The core idea is to have two versions of the same model: a "student" (the model being trained) and a "teacher" (an exponential moving average, or EMA, of the student's weights).

During training, the system takes a clean text sequence and creates two noisy versions of it along the same interpolation path: a noisier version for the student and a cleaner version for the teacher. Because the teacher sees less noise, it solves an easier problem, and its internal states and predictions serve as a "guide" for the student.

The authors propose two main variants:

  • SERF-L (Latent Distillation): The student’s early hidden layers are trained to match the teacher’s deeper hidden layers. This encourages the model to form high-level contextual representations much earlier in its processing pipeline.
  • SERF-P (Probability Distribution Distillation): The student’s predicted token distribution is trained to match the teacher’s distribution. A specific sub-variant, SERF-Pk, applies this supervision to an early student layer using a small adapter.
Overview of the SERF framework showing Latent and Probability Distribution Distillation variants.

Datasets

The researchers evaluated SERF across three distinct continuous language-model paradigms using:

  • OpenWebText (OWT): A large-scale corpus of web pages for unconditional text generation.
  • TinyGSM / GSM8K: A dataset of grade-school math word problems used to evaluate reasoning capabilities.

Key Insights and Ideas

  • The Trajectory is the Teacher: You don't need an external model to improve diffusion training. The model's own future self (the EMA teacher) at a cleaner noise level provides a rich, internal supervision signal.
  • Early-to-Late Alignment is Key: One of the most effective strategies was aligning early student layers with late teacher layers. This forces the student to "anticipate" the final clean representation early on, leading to much more efficient learning.
  • Distribution-Level Supervision: For language, matching the teacher's predicted probability distribution (SERF-P) is often more natural and effective than just matching raw feature vectors, as it aligns directly with the cross-entropy objective used in standard language modeling.
  • Efficiency Gains are Real: Even though SERF requires an extra forward pass during training (adding about 30% overhead per step), the model converges so much faster that the total "wall-clock" training time is slashed by up to 7.7x.

Key Results

SERF demonstrates massive improvements across the board compared to baselines like FLM and S-FLM:

  • Training Acceleration: Achieves the same quality as the baseline with 7.7x less wall-clock training time on OpenWebText.
  • Generation Quality: Improves the MAUVE score (a measure of how close generated text is to human text) by 24.3%.
  • Reasoning Accuracy: Boosts GSM8K math accuracy by up to 28.8%.
  • Sampling Speed: Perhaps most impressively, SERF matches the performance of a 1024-step baseline using as few as 16 to 32 sampling steps—a 32x to 64x acceleration in inference speed.
Figure 1: Performance and efficiency gains of SERF. Improvements over FLM (Lee et al., 2026) on OpenWebText (Gokaslan & Cohen, 2019) and over S-FLM (Deschenaux & Gulcehre, 2026) on GSM8K. Left: SERF reaches the baseline’s quality with substantially fewer training steps (see Fig. 4), achieving wall-clock acceleration despite its small additional computational cost (see Sec. A.4). Middle: Improves generation quality and reasoning accuracy at the same sampling budget (see Tab. 1 and Fig. 5). Right: Matches the baseline’s 1024-step performance with only a fraction of the sampling steps (see Tab. 1, Fig. 4 and Fig. 8). Sources for each value are listed in Tab. 2.

Limitations and Future Work

The current work focuses exclusively on continuous flow-based and diffusion models. While these are gaining traction, many popular diffusion models for language still operate in discrete space (using masking or token replacement). The authors note that extending the SERF framework to these discrete models is a promising next step, though it may require different design considerations for the teacher supervision mechanism.

Additionally, while the training is faster overall, the per-step computational cost is higher due to the teacher's forward pass. Future work could explore ways to mitigate this overhead further.

Summary updated

Abstract

We introduce SERF (Self-Regularized Flow Language Models), a simple self-supervised framework for improving continuous diffusion language models by exploiting the denoising trajectory itself. While continuous flow-based language models offer a promising alternative to autoregressive generation, their training objectives typically rely on supervision from clean target tokens, leaving the denoising trajectory underexploited as a learning signal. SERF addresses this limitation through noisy-to-clean self-distillation: an exponential moving average (EMA) teacher receives a cleaner version of the student's noisy input along the same interpolation path and provides additional supervision. We investigate two variants: SERF-L, which aligns the student's intermediate representations with deeper teacher representations, and SERF-P, which aligns their output token distributions, either at the student's output or at an early student layer. All variants augment the standard training objective without requiring external pretrained models or modifying the inference procedure. Experiments across three continuous language-model paradigms (one-hot FLM, latent ELF, and hyper-spherical S-FLM) demonstrate substantial improvements in training efficiency, generation quality, and few-step generation. In particular, SERF achieves up to 7.7x wall-clock training acceleration on OpenWebText, improves MAUVE by 24.3% and GSM8K accuracy by up to 28.8%, and matches baseline performance with up to 64x fewer sampling steps. These results establish noisy-to-clean self-distillation as an effective approach to improving continuous diffusion language models.
BibTeX
@misc{kouzelis2026self,
  author = {Theodoros Kouzelis and Spyros Gidaris and Nikos Komodakis},
  title = {{Self-Regularized Flow Language Models}},
  year = {2026},
  eprint = {2610.37115},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37115},
}
Figure