Abstract
As generative artificial intelligence (AI) becomes a cornerstone of digital content creation, the internet is increasingly populated by synthetic data. This paper investigates the phenomenon of "Recursive Modality Changes" (RMC), where content is iteratively transformed between visual and textual modalities using state-of-the-art models like GPT-4o and DALL-E 3. Our findings indicate that these multimodal loops do not reach a stable equilibrium. Instead, they exhibit a consistent divergence, where original subjects are lost to semantic drift and stylistic "hallucinations," posing challenges for the long-term reliability of AI-generated ecosystems.
Introduction
The concept of "model collapse"—the degradation of AI performance when trained on its own output—is well-documented in unimodal contexts like text generation. However, the dynamics of multimodal recursion remain less understood. This study explores whether a "soul" or core identity of content survives when passed through a multimodal game of "Telephone." By repeatedly describing images and regenerating them from those descriptions, we test if the process converges on a stable "fixed point" or drifts into chaos.
Methodology: Recursive Modality Changes (RMC)
The researchers formalized a loop known as Recursive Modality Changes (RMC). The process begins with an initial image ($Image_0$) sourced from the COCO dataset. GPT-4o is tasked with generating a detailed caption ($Description_0$), which is then fed into DALL-E 3 to generate a subsequent image ($Image_1$). This cycle is repeated for 40 iterations.
To quantify the drift, the study utilized LPIPS for perceptual similarity, BERT and TF-IDF for semantic text similarity, and BLIP for image-text alignment. The experiments covered 60 images across categories such as living things, vehicles, and inanimate objects.
Results and Divergence
The primary finding is that divergence is the rule, not the exception. The AI loop fails to maintain a stable representation of the original input. For instance, an image of a person playing with a frisbee on a beach gradually transforms into a series of increasingly abstract, psychedelic cosmic mandalas.
Similarity Decay and Subject Loss
LPIPS similarity scores showed a sharp decline within the first 10 generations, eventually plateauing at a level indicating near-zero perceptual resemblance to the source. By the 39th generation, the original category (e.g., "person") was preserved in only 40% of the test cases. Interestingly, the study found that most information loss occurs during the text-to-image phase (DALL-E 3), where the model introduces creative flourishes that steer the trajectory away from the original intent.
Analysis of Stability Factors
Despite the overall trend of divergence, certain factors influenced the rate of change:
- Stylistic Anchors: DALL-E 3’s "vivid" setting acted as a stabilizer compared to the "natural" setting. The high-contrast, dramatic style provided a consistent aesthetic framework that slowed semantic drift.
- Category Persistence: Some objects proved more resilient. "Toilets" maintained their identity in 70% of generations, while "elephants" and "fire-hydrants" were frequently lost or transformed.
- Cyclical Behavior: Trajectories often exhibited "mini-stable" cycles, remaining relatively consistent for approximately six generations before a significant "mutation" occurred, resetting the path of divergence.
Conclusion
The study demonstrates that multimodal AI loops are inherently unstable. The tendency for models to prioritize creative generation over faithful reconstruction leads to a cumulative loss of information. As AI-generated content continues to proliferate, understanding these divergence patterns is crucial for maintaining the integrity of digital information and preventing the "hallucinatory" drift of the web's visual landscape.
Summary updated