Laminating Representation Autoencoders for Efficient Diffusion

Ramón Calvo-González, François FleuretarXiv 2026 · DOI · arXiv

Abstract

Recent work has shown that diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents. However, the dense patch grids from encoders like DINOv2 contain significant redundancy, making diffusion needlessly expensive. We introduce FlatDINO, a variational autoencoder that compresses this representation into a one-dimensional sequence of just 32 continuous tokens—an 8x reduction in sequence length and 48x compression in total dimensionality. On ImageNet 256x256, a DiT-XL trained on FlatDINO latents achieves a gFID of 1.80 with classifier-free guidance while requiring 8x fewer FLOPs per forward pass and up to 4.5x fewer FLOPs per training step compared to diffusion on uncompressed DINOv2 features. These are preliminary results and this work is in progress.

Introduction

Standard Latent Diffusion Models (LDMs) typically operate on latent spaces derived from pixel-space Variational Autoencoders (VAEs). Recent research has demonstrated that diffusing directly on self-supervised learning (SSL) features—specifically those from DINOv2—can lead to faster convergence and superior semantic alignment.

However, DINOv2 produces a dense 2D grid of patch embeddings (e.g., 256 patches for a $256 \times 256$ image). This grid contains massive spatial redundancy because neighboring patches often share the same semantic information. Processing this full grid through a Diffusion Transformer (DiT) is computationally prohibitive. FlatDINO addresses this by "flattening" dense feature grids into a compact 1D sequence, eliminating redundancy without sacrificing the reconstructibility or "diffusability" of the features.

The FlatDINO Architecture

The core of this approach is a variational autoencoder designed specifically to compress SSL features into a 1D latent space:

  1. Encoder (ViT-B): The encoder receives 256 DINOv2 patch embeddings. It prepends $T=32$ learnable "register" tokens to the input. After processing, the original 256 patches are discarded, and only the 32 register tokens are retained as the latent representation.
  2. Latent Space: The resulting representation is a 1D sequence of 32 continuous tokens, representing an 8x reduction in sequence length and a 48x reduction in total dimensionality compared to the raw DINOv2 output.
  3. Decoder (ViT-L): The decoder uses learnable queries to attend to the 32 latent tokens and reconstruct the original 256 DINOv2 patch embeddings.
  4. Generative Modeling: A Diffusion Transformer (DiT-XL) is trained using Flow Matching directly on this compressed 32-token space.
The FlatDINO architecture compresses dense DINOv2 feature grids into a compact 1D latent sequence using a ViT-based variational autoencoder.

Key Insights: Spatial Redundancy and Token Specialization

Analysis of DINOv2 features reveals that while they have little linear redundancy (PCA requires nearly 600/768 dimensions to explain 95% variance), they possess high spatial redundancy. This justifies compressing the number of tokens rather than just the dimensionality of each token.

Token ablation studies show that FlatDINO learns a form of spatial partitioning. Most of the 32 tokens specialize in localized "blobs" or regions of the image. Interestingly, the model exhibits a "phase transition" when reducing the token count below 32; at 16 tokens, the model shifts from localized blobs to horizontal stripes, which correlates with a sharp drop in reconstruction quality. This suggests 32 tokens is a critical threshold for maintaining sufficient spatial coverage.

Furthermore, this compression acts as a semantic filter. Compressing features to 32 tokens improved k-NN accuracy on ImageNet from 74.3% to 77.2%, suggesting the bottleneck effectively removes noise while preserving core discriminative information.

Experimental Results

FlatDINO demonstrates significant efficiency gains across both training and inference. It achieves an 8x reduction in FLOPs per forward pass during inference. During training, even when accounting for the encoder overhead, it requires 4.1x to 4.5x fewer total FLOPs per step compared to diffusing on uncompressed features.

Comparison of inference efficiency (GFLOPs) and generation quality (gFID) on ImageNet 256x256. FlatDINO achieves significant compute savings while maintaining competitive quality.

In terms of generation quality, a DiT-XL trained on FlatDINO latents achieved a gFID of 1.80 on ImageNet $256 \times 256$ (with classifier-free guidance). While this is slightly behind the state-of-the-art for uncompressed features (gFID 1.13), it represents a highly competitive result given the massive reduction in computational requirements.

Conclusion and Future Work

FlatDINO provides a path toward highly efficient high-resolution diffusion by leveraging the spatial redundancy of SSL features. While these preliminary results are promising, a small quality gap remains compared to uncompressed methods. Future work will explore longer training schedules, joint optimization of the autoencoder and diffusion model, and refining the decoding process to eliminate color distortions observed on out-of-distribution data.

Summary updated

Abstract

Recent work has shown that diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents. However, the dense patch grids from encoders like DINOv2 contain significant redundancy, making diffusion needlessly expensive. We introduce FlatDINO, a variational autoencoder that compresses this representation into a one-dimensional sequence of just 32 continuous tokens -an 8x reduction in sequence length and 48x compression in total dimensionality. On ImageNet 256x256, a DiT-XL trained on FlatDINO latents achieves a gFID of 1.80 with classifier-free guidance while requiring 8x fewer FLOPs per forward pass and up to 4.5x fewer FLOPs per training step compared to diffusion on uncompressed DINOv2 features. These are preliminary results and this work is in progress.
BibTeX
@misc{calvogonzalez2026laminating,
  author = {Ramón Calvo-González and François Fleuret},
  title = {{Laminating Representation Autoencoders for Efficient Diffusion}},
  year = {2026},
  eprint = {2602.04873},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.04873},
}
Figure