Mitigating Over-Saturation and Over-Smoothing in Score Distillation

135 concepts102 papers1 proposal

Score Distillation Sampling (SDS) has become the standard engine for "lifting" 2D diffusion priors into 3D representations. By using a pre-trained image diffusion model as a critic, researchers can optimize 3D structures—such as Neural Radiance Fields (NeRF) or 3D Gaussian Splats—to match text descriptions without requiring massive 3D datasets. However, this process is notoriously prone to two distinct visual artifacts: over-saturation (a "deep-fried," unnaturally vibrant look) and over-smoothing (a "melted" appearance lacking high-frequency textures like pores or fabric weaves).

Understanding why these artifacts occur requires looking at how the distillation gradient interacts with the manifold of natural images.

The Mode-Seeking Problem

The primary cause of over-smoothing is the "mean-seeking" or "mode-seeking" nature of the SDS loss function. As noted by Zhou et al. in UltrAvatar, SDS tends to collapse the diverse distribution of a diffusion model into a single, simplified average.

When a 3D model is optimized against a diffusion prior, it receives update directions from multiple stochastic noise sources. Miao et al. argue in Rethinking Score Distilling Sampling for 3D Editing and Generation that this creates an "averaging effect." Because the optimization process pulls the 3D representation toward many possible "clean" versions of a noisy image simultaneously, high-frequency details and fine textures are washed out. Zhou et al. further explain in Zero-1-to-A that when generative signals are inconsistent across views, the optimization objective reconciles this variance by blurring the conflicting details, leading to the characteristic "melted" look.

The Guidance Scale Trade-off

To ensure that a 3D model actually resembles the text prompt, researchers typically employ very high Classifier-Free Guidance (CFG) scales. While high CFG improves structural coherence, it is the primary driver of over-saturation.

As Zhang et al. observe in HumanRef, standard SDS relies on these high scales to overcome noisy gradients, but this forces the optimization to over-emphasize certain features at the expense of natural color distributions. This results in "deep-fried" textures with exaggerated contrast and vibrant, plastic-like skin tones. Alldieck et al. demonstrate in Score Distillation Sampling with Learned Manifold Corrective that these artifacts occur because the optimization trajectory is not inherently constrained to the manifold of natural images. Without a corrective mechanism, the gradients push the pixels toward extreme values that the diffusion model considers "highly probable" under the prompt, even if they are aesthetically unrealistic.

Inconsistent Trajectories and Noise

A third facet of the problem lies in the randomness of the distillation process itself. Li et al. point out in FLOWDREAMER that randomly sampling noise during optimization leads to multiple inconsistent optimization trajectories for the same target. This lack of a deterministic path prevents the model from converging on sharp, high-fidelity details.

Similarly, Zhuo et al. analyze the SDS loss in VividDreamer by decomposing it into a reconstruction term and a guidance term. Their analysis reveals that the "single-step reconstruction jump"—the attempt to predict a clean image from a noisy one in one go—is mathematically responsible for the blurring seen in 3D textures.

Emerging Solutions

Recent work has focused on replacing the standard SDS loss with more stable alternatives:

  • Refined Gradients: Katzir et al. propose Noise-free Score Distillation (NFSD), which mathematically removes an unwanted noise term from the score, allowing for high-quality generation at lower guidance scales.
  • Multi-step Refinement: Instead of single-step jumps, Zhuo et al. use multi-step refinement derived from DDIM sampling to maintain texture integrity.
  • Manifold Constraints: Alldieck et al. introduce a learned corrective network to force the optimization to stay on the manifold of natural images, mitigating the "deep-fried" effect.
  • Curriculum Learning: Yi et al. suggest a Diffusion Time-step Curriculum (DTC123), scheduling the distillation from coarse to fine to prevent the model from getting stuck in over-saturated local optima early in training.

By addressing the underlying mathematical causes—mode-seeking behavior, manifold deviation, and stochastic inconsistency—these methods are moving 3D generation away from cartoonish approximations toward hyper-realistic digital assets.

Key Foundational Work

Guiding 3D Digital Content Generation with Pre-Trained Diffusion Models (2024) — contributed 3 concept elements to this direction.
DreamLCM: Towards High Quality Text-to-3D Generation via Latent Consistency Model (2024) — contributed 2 concept elements to this direction.
Score Distillation via Reparametrized DDIM (2024) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_score_distillation_artifacts_5c57e6fb,
  title = {score distillation artifacts},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/score-distillation-artifacts/rdr_5c57e6fb871f4608af2fc7bfb814eb97},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_score_distillation_artifacts_5c57e6fb_v1,
  title = {score distillation artifacts},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/score-distillation-artifacts/rdr_5c57e6fb871f4608af2fc7bfb814eb97/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake