TL;DR
Neural Preset is a novel color style transfer technique that achieves real-time 4K performance and artifact-free results by replacing standard convolutional mapping with Deterministic Neural Color Mapping (DNCM). By decoupling color normalization from stylization, it allows users to extract and reuse color styles as lightweight "presets," making high-resolution photo and video editing both efficient and visually consistent.
What Problem Does This Paper Address?
Color style transfer—mapping the aesthetic palette of one image onto another—is a staple of modern photo editing. However, existing deep learning methods suffer from a "triple threat" of practical limitations:
- Visual Artifacts: Convolutional neural networks (CNNs) operate on image patches. This spatial dependency often leads to inconsistent mapping for pixels of the same color, resulting in distorted textures and dissonant "blotchy" colors.
- Memory Bottlenecks: Most state-of-the-art (SOTA) models have massive memory footprints, frequently running into Out-of-Memory (OOM) errors when processing 4K or 8K images, even on high-end GPUs.
- Inefficient Style Switching: Traditional models are single-stage; to apply a new style, you must rerun the entire heavy inference pipeline.
Methods
The authors introduce two core innovations to solve these issues:
1. Deterministic Neural Color Mapping (DNCM)
Instead of relying on standard CNN layers for the final color transformation, the authors propose DNCM. As shown in Figure 2, DNCM uses a small encoder to predict an image-adaptive color mapping matrix ($T$). DNCM architecture Crucially, this mapping is deterministic: every pixel of a specific color in the input is mapped to the same specific color in the output. This eliminates spatial artifacts and allows the model to process pixels independently, drastically reducing memory usage.
2. Two-Stage "Neural Preset" Pipeline
The process is split into two distinct stages (see Figure 3):
- Normalization: An input image is mapped to a "normalized color style space" that represents only the image content.
- Stylization: The normalized content is then mapped to the target style using parameters ($r_s$) extracted from a style image. Two-stage pipeline This design allows the style parameters to be stored as tiny "presets" (only a few hundred parameters) and reused instantly across different images or video frames.
Self-Supervised Training
Because paired datasets (same content, different expert-retouched styles) are rare, the authors developed a self-supervised strategy. They apply random color perturbations (like filters or LUTs) to a single image to create two versions with the same content but different styles. They then train the model to ensure that both versions normalize to the same representation and can reconstruct each other when their style parameters are swapped (Figure 4). Self-supervised training
Datasets
- Training: The model was trained on the MS COCO dataset, augmented with ~5,000 different Look-Up Tables (LUTs) and random filter adjustments to simulate diverse color styles.
- Evaluation: The authors used a validation set of 50 images with diverse styles (2,500 pairs total). They also benchmarked on iHarmony4 (for image harmonization) and MIT-Adobe FiveK (for color enhancement).
- New Metric: They introduced a new style similarity metric by training a discriminator on a custom dataset of 700+ color style categories to better capture human aesthetic preferences compared to traditional metrics like VGG Gram matrices.
Key Insights and Ideas
- Determinism vs. Convolutions: The biggest "aha" moment is that for color transfer, spatial context (convolutions) is often the enemy. By forcing pixel-wise determinism, you guarantee texture preservation.
- Efficiency through Low-Rank Projection: DNCM embeds RGB pixels into a higher-dimensional $k$-space (where $k=16$) before applying the transformation matrix. This allows the model to capture complex, non-linear color relationships with very few learnable parameters.
- Universal Generalization: Because the model learns a fundamental mapping between color distributions, it naturally generalizes to tasks like low-light enhancement and dehazing without any task-specific fine-tuning.
Key Results
- Speed: Neural Preset is roughly 28× faster than previous SOTA methods like PhotoWCT2. It achieves 52 fps at 4K resolution on an Nvidia RTX 3090.
- Memory: It maintains a constant memory footprint of 1.96GB, allowing it to handle 8K images easily, while all competing deep learning methods failed due to OOM errors (Figure 1). Comparison of speed and artifacts
- Quality: In a user study, Neural Preset was ranked as the "Best" result in 61.28% of cases, far outperforming the second-place method (16.25%).
- Quantitative: As shown in Figure 6, Neural Preset sits closest to the "Ideal" point on the Content vs. Style similarity curve, balancing the two better than any other method. Quantitative comparison
Limitations and Future Work
- JPEG Artifacts: If an input image is heavily compressed, the global color mapping can sometimes amplify existing JPEG blockiness.
- Missing Colors: The model may struggle if the input image contains colors that are completely absent from the style image (e.g., trying to map a lush green forest style onto a desert photo).
- Local Adaptation: DNCM is a global mapping. It cannot currently distinguish between two different objects of the same color (e.g., a blue car vs. a blue sky) to apply different transformations to them. The authors suggest future work could involve incorporating user masks or local-adaptive regularizers to handle these complex cases.
Summary updated