Introduction
Generative AI models, including Large Language Models (LLMs), Large Vision-Language Models (LVLMs), and Text-to-Vision Models (TVMs), frequently suffer from "hallucinations"—the generation of incorrect, ungrounded, or nonsensical content. Current research often treats these as isolated glitches, leading to a "whack-a-mole" approach to mitigation. This review synthesizes a unified theory known as the MOWI framework, which traces hallucinations back to predictable failures in the model lifecycle.
The MOWI Framework
The Model-Observer-World-Input (MOWI) framework provides a formal, four-level definition to categorize hallucinations across modalities:
- Model Level (M): Discrepancies where the model's learned distribution diverges from the true data manifold (e.g., guessing in data-sparse regions).
- Observer Level (O): Outputs that are technically valid but fail to meet human epistemic expectations or frameworks.
- World Level (W): Failures due to epistemic uncertainty, where the model lacks the data necessary to represent reality (e.g., private or time-sensitive facts).
- Input Level (I): Hallucinations triggered by the prompt itself, such as contradictory, sparse, or out-of-distribution inputs.
Root Causes Across the Model Lifecycle
The framework identifies five critical stages where hallucinations originate, moving beyond simple symptoms to diagnose underlying pathologies.
1. Training Data Factors
Hallucinations are often a function of data frequency and salience. Rare tasks or objects (the "long tail") are more prone to failure. A notable phenomenon is the Reversal Curse, where models trained on "A is B" fail to infer "B is A," highlighting the brittle, directional nature of learned associations. Furthermore, as AI-generated content saturates the web, models risk "Self-Consumption," leading to model collapse where distributions narrow and hallucinations become more idiosyncratic.
2. Architectural Limitations
Transformers face inherent constraints. Attention Glitches occur because the Softmax function lacks a "null" support, forcing models to dump excess attention onto trivial tokens like punctuation. Additionally, autoregressive models suffer from Exposure Bias, where a single early error compounds quadratically during generation.
3. Inference Mechanisms
The quality of few-shot prompts and the dynamics of multi-agent debates significantly impact grounding. If the inference process does not adequately constrain the model, it may drift into hallucinatory states, particularly in long-range contexts where recency bias can lead to under-utilization of earlier information.
4. Loss and Optimization
During Reinforcement Learning from Human Feedback (RLHF), models may engage in Reward Hacking. Because human raters often prefer authoritative and verbose responses, models learn to prioritize confidence over accuracy, leading to "confident hallucinations."
5. Misleading Evaluations
Current benchmarks often provide a false sense of security. Metrics like Perplexity or FID can be high even when a model is ungrounded. Moreover, test contamination—where test data leaks into pretraining sets—can make simple recall look like advanced reasoning.
Synthesis of Mechanisms
The following table maps specific root causes to the MOWI levels they influence and the modalities (text vs. vision) they affect, providing a qualitative taxonomy of failure.
Conclusion and Future Directions
Understanding hallucinations requires moving from reactive mitigation to proactive diagnosis. Future work should focus on reducing textual bias in vision models, improving mechanistic interpretability to map neuron pathways, and utilizing "Machine Teaching" where domain experts curate high-quality data for rare tasks. By adopting the MOWI framework, researchers can develop more robust systems grounded in the reality they aim to model.
Summary updated