Generative models are designed to be mirrors. Whether they are Diffusion Models or GANs, their objective is to capture and replicate the underlying distribution of their training data. However, these mirrors are often too faithful. They do not distinguish between the "signal"—the cat, the human voice, or the medical pathology—and the "noise"—the watermarks, the sensor grain, or the compression artifacts.
This phenomenon, often called artifact inheritance, means that generative models frequently internalize and reproduce undesirable features deeply embedded in their training sets.
The Mirroring of Non-Semantic Noise
When we train a model on millions of web-scraped images, we assume it is learning the concept of "a sunset." But as Runhui Huang and colleagues point out in DiffDis, these models also memorize text overlays and watermarks. Because these patterns appear consistently across large subsets of data (like stock photo collections), the model treats them as inherent features of high-quality imagery.
This isn't limited to visual watermarks. In the realm of audio, Heng Lu notes in UniSpeaker that models can capture and replicate background acoustic noise. Similarly, Hang Zhou found in Vision-Infused Deep Audio Inpainting that models often learn to replicate the specific background hum of the recording environment rather than synthesizing clean audio.
The danger is most acute in specialized fields like medicine. Alceu Bissoto and his team demonstrated in their review of skin-lesion GANs that models inadvertently learn to generate rulers, hair, and skin-gel bubbles. If these synthetic images are then used to train classifiers, the classifiers may learn to identify "cancer" based on the presence of a ruler rather than the lesion itself.
Why Models Struggle to "Clean" Data
A common intuition is that a model should be able to "see through" the noise to find the clean subject. However, Shaozhe Hao argues in ArtiFade that when a model learns from "dirty" data, the artifacts become "baked into" the subject's embedding. Because the model lacks a mechanism to decouple the subject from its environment, the noise becomes a permanent part of the subject's digital identity.
This is exacerbated by condition leakage. In tasks like image compression or super-resolution, the model is given a "bad" version of an image and asked to make it "good." As Yichong Xia explains in DiffPC, models often inadvertently reproduce the blurry or blocky artifacts of the input. Instead of synthesizing new, clean textures, they simply generate "sharpened" versions of the original distortions.
The Evaluation Trap: Gaming the Metrics
Artifact inheritance doesn't just ruin the output; it can also trick our evaluation metrics. The Fréchet Inception Distance (FID) is the gold standard for measuring how "real" a generated image looks. However, Gaurav Parmar and Richard Zhang discovered that FID is highly sensitive to low-level aliasing and JPEG compression noise.
In On Buggy Resizing Libraries, they show that if a model learns to replicate the specific JPEG artifacts of its training set, its FID score will improve. The metric interprets the matching noise distributions as a sign of "realism," even if the image quality is objectively lower. This creates a perverse incentive where models are rewarded for being "dirty" in the same way as their training data.
Strategies for Disentanglement
Researchers are developing several ways to break this cycle of inheritance:
- Manifold Attraction: Dennis Elbrächter proposes Manifold Attracted Diffusion (MAD), which uses the "manifold hypothesis"—the idea that clean data lives on a lower-dimensional surface than noisy data. During inference, MAD "attracts" the generated samples toward this clean manifold, effectively filtering out noise even if the model was trained on corrupted data.
- Implicit Concept Removal: Instead of trying to clean the data, some researchers try to "scrub" the model. Kai Chen introduced Geom-Erasing in Implicit Concept Removal of Diffusion Models, which uses geometric data to identify where watermarks or artifacts usually appear and explicitly removes those "concepts" from the model's generation process.
- Negative Preference Optimization: Fu-Yun Wang and colleagues suggest Diffusion-NPO, which trains a "negative" model to learn what users dislike (like artifacts or poor lighting). During generation, the model is steered away from these negative traits.
By understanding that generative models are indiscriminate learners, the field is moving from simply "scaling up" data to "cleaning up" the relationship between the model and its source material.