Abstract: Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public. It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images. In this paper we consider what the future might hold. What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. We refer to this effect as Model Collapse and show that it can occur in Variational Autoencoders, Gaussian Mixture Models and LLMs. We build theoretical intuition behind the phenomenon and portray its ubiquity amongst all learned generative models. We demonstrate that it has to be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in data crawled from the Internet.
1. Introduction
We are currently in a "gold rush" for Large Language Models (LLMs), with models like GPT-4 trained on massive scrapes of the internet. However, as the digital ecosystem becomes increasingly populated with AI-generated text and images, a looming systemic risk emerges: Model Collapse. This paper investigates the consequences of models "eating their own tail"—training on the output of their predecessors. Unlike catastrophic forgetting or data poisoning, Model Collapse is an inevitable degenerative process where errors in statistical sampling and model approximation compound over generations, leading to an irreversible loss of information about the true underlying data distribution.
2. The Mechanism of Model Collapse
Model collapse is driven by two primary sources of error that manifest during recursive training. First, Statistical Approximation Error occurs due to finite sampling; if rare events (the "tails" of a distribution) are not captured in one generation's sample, the subsequent model will never learn of their existence. Second, Functional Approximation Error arises from the inherent limitations of neural networks, which may fail to capture the full complexity of the data or assign probability to "nonsense" regions.
As illustrated in the figure above, the process creates a feedback loop. Probable events begin to "poison reality" by becoming over-represented, while improbable events are progressively filtered out. This results in a "random walk" where model parameters drift further from the true distribution over time.
3. Methodology
The researchers evaluated this phenomenon across three distinct model classes:
- Gaussian Mixture Models (GMMs): To provide a low-dimensional analysis of distribution shifts.
- Variational Autoencoders (VAEs): Tested on the MNIST dataset to visualize latent representation evolution.
- Large Language Models (LLMs): Specifically fine-tuning the OPT-125m model on the WikiText-2 dataset.
The core methodology involved recursive fitting-sampling: training Model $n$ on data from Model $n-1$, generating a new dataset, and repeating the process while measuring the divergence from the original human-generated ground truth.
4. Results and Discussion
The experiments revealed a consistent pattern of decay. In "early" model collapse, the model loses information about rare events at the edges of the distribution. In "late" collapse, the distribution converges to a narrow mean with near-zero variance.
Key findings include:
- LLM Performance Decay: Fine-tuning OPT-125m on its own generations caused perplexity to rise from ~20 to ~28. Qualitatively, models began producing repetitive gibberish, such as an obsession with "jackrabbits" regardless of the prompt context.
- VAE Degradation: By the 20th generation, VAE reconstructions of MNIST digits became indistinguishable, blurring different numbers into a single, unimodal average shape.
- Data Preservation: While keeping 10% of original human data in the training set slowed the collapse, it did not prevent the eventual degradation of the model's perception of reality.
5. Conclusion
Model collapse highlights a significant "first mover advantage" for models trained on the pre-AI internet. As the web becomes polluted with synthetic content, the value of original, human-produced data—such as books and genuine forum interactions—will skyrocket. To sustain the benefits of generative AI, future research must prioritize data provenance, watermarking, and the development of training techniques that are robust to the compounding errors of recursive data loops.
Summary updated