TL;DR
While LLM hallucinations are usually treated as bugs to be squashed, this paper argues they can be a powerful source of "computational imagination" for creative professionals. By analyzing real-world interactions, the authors present a framework for how practitioners judge, adapt, and "salvage" inspiration from AI errors, transforming incorrect outputs into novel creative directions.
What Problem Does This Paper Address?
In the world of Large Language Models (LLMs), "hallucination"—generating factually incorrect or instruction-defying content—is typically seen as a failure of reliability. However, for creative practitioners (designers, writers, researchers), an "error" is sometimes just an unexpected idea.
Prior research has noted that hallucinations can spark creativity, but we haven't understood the mechanics of this process. Why is one fabricated reference annoying while another is "eye-opening"? This paper tackles the gap between error detection and creative utility, exploring how practitioners navigate the tension between an LLM's unreliability and its potential as a "provocateur" in the creative process.
Methods
The authors conducted a retrospective qualitative study with 20 experienced LLM users across three domains: Research, Design, and Social Media Content Creation.
The study used a two-step approach:
- Color-Coding Task: Participants reviewed their own chat logs (from ChatGPT, DeepSeek, etc.) and identified 41 specific hallucination cases. They coded these as Green (prompted action/exploration), Yellow (interesting but not pursued), or Red (meaningless or unacceptable).
- Semi-Structured Interviews: The researchers interviewed participants to understand the "why" behind their judgments, how they responded to these errors, and how their trust in the AI shifted as a result.
The analysis distinguished between factual hallucinations (conflicting with real-world facts) and faithfulness hallucinations (violating user instructions or logic).
Datasets
This study did not use a standard machine learning benchmark. Instead, it utilized a qualitative dataset of 41 real-world hallucination cases volunteered by the participants from their actual work history.
Key Insights and Ideas
- Inspiration is Stage-Dependent: Hallucinations are far more likely to be seen as "inspiring" during early-stage ideation and brainstorming. During the execution or "finalizing" phase, the same errors are viewed as high-risk failures.
- The "Salvage" Mindset: Practitioners often perform a "creative extraction." They might recognize a fact is false (e.g., a fabricated historical event) but keep the "creative structure" or "logic" of the error to use in a fictional script or a design concept.
- Role-Differentiated Trust: Hallucinations don't just "lower trust" across the board. Instead, they cause users to "re-calibrate" what they trust the LLM for. A user might stop trusting the LLM as a "source of facts" but increase their trust in it as a "divergence partner" or "fuzzy search tool."
- Core vs. Subjective Constraints: For instruction-following (faithfulness), inspiration only happens when the AI violates "subjective" preferences (like style) while respecting "core" constraints (like safety or fundamental project requirements).
Key Results
The paper introduces a Framework from Hallucination to Inspiration (summarized in Figure 1 of the paper), which maps three distinct routes for factual hallucinations:
- Route 1: The user is inspired by the possibility before even verifying if it's true.
- Route 2: The user tries to verify a fake claim and, in the process of searching, discovers real related materials that spark a new idea.
- Route 3: The user knows it's fake but retains the "unexpected creative elements" anyway.
The authors also propose several design recommendations for future LLMs:
- Stage-Sensitive Control: An "Inspiration Mode" that allows for higher divergence during brainstorming but tightens adherence during execution.
- Salvage Support: UI tools that allow users to "pin" a useful part of a hallucination while asking the model to correct the factual errors around it.
- Differentiated Labeling: Moving beyond "this might be wrong" to labels like "speculation," "creative hypothesis," or "low-confidence inference."
Limitations and Future Work
The study's primary limitation is its small, specialized sample size (20 young professionals), which may not represent how general users or professionals in high-stakes fields (like medicine or law) interact with AI. Additionally, the study relies on retrospective recall, which can be subject to memory bias.
Future work suggests implementing the "salvage support" and "stage-sensitive control" features in real-world LLM interfaces to see if they actually improve creative outcomes while mitigating the risks of misinformation.
Summary updated