D-LUT: Photorealistic Style Transfer via Diffusion Process

Mujing Li, Guanjie Wang, Xingguang Zhang, Qifeng Liao, Chenxi XiaoWACV 2025 · DOI

Abstract

This paper introduces D-LUT, a novel framework for photorealistic color style transfer that utilizes diffusion processes to generate 3D Look-Up Tables (LUTs). By treating color transfer as a distribution alignment problem solved via score matching and Langevin dynamics, D-LUT achieves artifact-free, high-fidelity results. The method is computationally efficient, capable of processing Full HD video at 90 FPS, and produces industry-standard LUT files compatible with professional editing software.

Introduction

Professional color grading is essential for high-quality visual media, yet automating this process while maintaining photorealism remains a significant challenge. Traditional deep learning approaches, typically based on encoder-decoder architectures, often introduce visual artifacts such as noise or halos because they reconstruct images from scratch. Furthermore, these models are frequently too computationally heavy for real-time applications and lack portability to standard industry tools like Adobe Premiere or DaVinci Resolve.

D-LUT addresses these hurdles by bridging the gap between generative AI and professional workflows. Instead of generating a new image directly, D-LUT generates a 3D Look-Up Table (LUT)—the industry standard for deterministic, pixel-wise color mapping. This approach ensures that the original image's structural integrity and sharpness are preserved while its color palette is transformed to match a target style.

Methodology: Diffusion-Based LUT Generation

The core innovation of D-LUT is the use of a diffusion-based score-matching algorithm to align the color distribution of a content image with that of a style image.

The D-LUT pipeline: User-preferred style images are used to train a 3D LUT generator, which then performs color transfer.

The process follows several key stages:

  1. Score Matching: A compact Multi-Layer Perceptron (MLP) with approximately 93K parameters is trained on-the-fly to estimate the "score" (the gradient of the log-probability density) of the style image's color palette. This allows the model to learn the specific "shape" of the target colors.
  2. Langevin Dynamics: Using the learned score, color points from the content image are iteratively "diffused" toward the high-density regions of the style palette.
  3. LUT Extraction: To maintain efficiency, the algorithm diffuses the points of a 3D identity lattice rather than every pixel in the image. By recording the final positions of these lattice points, a 3D LUT is created.
  4. Averaging for Stability: To mitigate the inherent randomness of Langevin noise, the process is run multiple times and the resulting LUTs are averaged, ensuring a smooth and stable transformation.

Results and Discussion

D-LUT demonstrates superior performance across speed, memory usage, and visual quality compared to state-of-the-art models such as WCT2 and PhotoWCT2.

A retouched landscape image demonstrating the high-quality, artifact-free color transfer achieved by D-LUT.

Performance and Efficiency

  • Real-Time Capability: D-LUT achieves an inference time of 0.011s for Full HD images on a GTX 1080 Ti, enabling real-time video processing at roughly 90 FPS.
  • Low Resource Footprint: The model utilizes only 1.36GB of GPU memory, a fraction of the requirements for traditional deep learning stylization models (e.g., 18.75GB for WCT2).
  • Portability: The resulting .CUBE files are fully explainable and portable, allowing editors to fine-tune the results in professional software.

Visual Fidelity and Consistency

Because D-LUT outputs a LUT rather than a reconstructed image, it is mathematically guaranteed to preserve sharpness and fine details. Qualitative evaluations show that D-LUT avoids the blurring of fine textures common in other methods. Furthermore, applying the same LUT to every frame of a video ensures perfect temporal consistency, eliminating the flickering artifacts often seen in AI-driven video stylization.

Conclusion

D-LUT provides a robust, efficient, and industry-compatible framework for photorealistic style transfer. By leveraging diffusion processes to generate standard 3D LUTs, it offers a practical tool for professional color grading. Future work may explore the use of segmentation masks to enable localized color grading and the development of automatic template selection to improve results for mismatched content-style pairs.

Summary updated

BibTeX
@inproceedings{li2025d,
  author = {Mujing Li and Guanjie Wang and Xingguang Zhang and Qifeng Liao and Chenxi Xiao},
  title = {{D-LUT: Photorealistic Style Transfer via Diffusion Process}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00892},
}
Figure