LABridge: Text–Image Latent Alignment Framework via Mean-Conditioned OU Process

TL;DR

LABridge introduces a novel framework that replaces the generic "one-size-fits-all" noise starting point of traditional diffusion models with a structured, text-aligned prior. By utilizing a mean-conditioned Ornstein–Uhlenbeck (OU) process, the model creates a "mean-reverting" path that pulls the generation process toward the desired text semantics. This results in faster sampling, better stability, and significantly improved text-to-image alignment.

The Problem with Standard Diffusion

Standard diffusion models typically start their generative process from a homogeneous Gaussian distribution ($\mathcal{N}(0, \mathbf{I})$). This approach presents three major challenges:

1. Semantic Instability: Because the starting point is the same for every prompt, the model must "disentangle" vastly different concepts (like "cat" vs. "landscape") from the identical unstructured noise. This overlap can lead to ambiguity and "hallucinations."
2. Inefficient Sampling: Without strong initial guidance, the denoising path is often curved and complex, requiring many iterative steps (high NFEs) to reach a high-fidelity image.
3. Weak Text-Vision Alignment: The connection between the text prompt and the image manifold is often indirect, making it difficult for models to accurately capture nuanced or complex prompts.

Expand

Source, p. 2

Methodology: The LABridge Framework

To overcome these limitations, the authors propose the Latent Alignment Framework (LABridge), which explicitly preserves and aligns textual and visual semantics in a shared latent space. The framework consists of two main components:

Expand

Source, p. 5

1. Text-Image Alignment Encoder (TIAE)

The TIAE is a new component designed to bridge the gap between text and vision. It takes text embeddings (from models like CLIP or T5) and maps them to a specific "prior mean" ($\mu_T$) within the image latent space. It is trained using a composite loss:

• Latent Alignment Loss: Forces the text prior to be close to the actual image latent.
• Semantic Consistency Loss: Ensures that semantically similar texts (e.g., "dog" and "puppy") map to nearby points in the latent space.
• Reconstruction Loss: Ensures the generated prior can actually be decoded into a meaningful image.

2. Mean-Conditioned OU Process

Instead of standard Gaussian diffusion, LABridge uses an Ornstein–Uhlenbeck (OU) process. In physics, this process describes a particle that is "pulled" back toward a central mean. LABridge uses the TIAE's output as this central mean, providing a "directed drift" that guides the image generation process more directly toward the target semantics from the very first step.

Key Insights

• Stop Starting from Scratch: The core insight is that starting from pure, unstructured noise is unnecessary. If the goal is to generate a "cat," the process should begin in a region of the latent space that already structurally aligns with a cat.
• Directed Drift: By using the OU process, the model doesn't just blindly denoise; it is actively "pulled" toward the text-conditioned mean. This makes the sampling trajectory straighter and more certain, naturally accelerating generation.
• Preserving Semantic Topology: The Semantic Consistency Loss ensures that the "logic" of the text embedding space is preserved in the image prior space, helping the model handle nuanced or out-of-distribution prompts better.

Experimental Results

LABridge was trained on a selection from the COYO-700M dataset and evaluated across multiple popular backbones, including Stable Diffusion 1.5, SDXL, SD 3.5, and PixArt-$\alpha$.

• Improved Text Alignment: On the GenEval benchmark, LABridge improved the score of SD 3.5-Large from 0.66 to 0.69. On DPG-Bench, it improved from 84.52 to 85.28.
• Better Image Quality: On the MJHQ-30K dataset, LABridge achieved a significantly lower (better) FID score of 3.82 compared to the base SD 3.5-L score of 5.84.
• Efficiency: The framework achieves competitive results with fewer denoising steps because the "informed initialization" starts the process much closer to the final target.
• Backbone Agnostic: The TIAE can be plugged into various existing architectures, showing consistent improvements in both image fidelity and text consistency.

Limitations and Future Work

While highly effective, LABridge requires a sequential two-stage training process (Stage 1 for TIAE, Stage 2 for the OU process), which is more complex than standard end-to-end training. Additionally, its performance remains somewhat bounded by the quality of the frozen VAE and text encoders used. Future work may explore extending this latent alignment strategy to other modalities, such as text-to-video or text-to-3D, where semantic stability is even more challenging to maintain.

@inproceedings{shao2025labridge,
  author = {Huiyang Shao and Xin Xia and Yuxi Ren and XING WANG and Xuefeng Xiao},
  title = {{LABridge: Text–Image Latent Alignment Framework via Mean-Conditioned OU Process}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=CABcYH1wKM},
}