The "gap-based ideation" recipe is a staple of AI-driven creativity hype: take a massive dataset (like all games on Steam), map them into an embedding space, find the "holes" where no data exists, and decode those coordinates into a brilliant new idea. It sounds mathematically sound, but does it actually work?
In a rigorous new paper, Javier Aguilar Martín puts this premise to the test. By using temporal splits—building maps from "past" data and checking them against "what came next"—the author finds that embedding distances are a poor guide for ideation. In short: the voids in our maps are usually empty for a reason, and they aren't hiding the next big thing.
TL;DR
This paper systematically debunks "gap-based ideation," the theory that "empty" spots in an embedding map represent novel, viable ideas. Through four pre-registered tests, the author shows that future work follows existing trends ("momentum") rather than filling gaps, and that LLMs struggle to generate coherent text that actually lands in these distant, empty regions.
What Problem Does This Paper Address?
As Large Language Models (LLMs) are increasingly used for scientific discovery and product design, researchers have sought ways to steer them toward "novelty." A popular approach is to use the geometry of embedding spaces—mathematical representations of meaning—to identify "content gaps."
The assumption is that distances in these maps carry information about where "good" new work lies. If we can find a coordinate far from existing work, we’ve found a "blue ocean" of opportunity. This paper asks if that assumption holds up when compared to actual historical data. Do the "holes" in a 2023 map of arXiv papers actually get filled by the breakthrough papers of 2024?
Methods
The author designed four tests to challenge the "gap" hypothesis:
- Decoding the Voids: Can an LLM actually write coherent text that corresponds to a "far" random point in vector space?
- Forecasting Growth: Do sparse regions of a map (gaps) grow faster in the future than dense regions?
- Steering Continuations: Can distance-based feedback help an LLM move "further" from a seed paper toward real-world follow-up research?
- Inspiration Distance: Is there an "optimal" distance for inspiration material (the "Inverted U" hypothesis)?
The study utilized a "past-only" map to ensure no data leakage and employed several state-of-the-art models (GPT-4o, Claude 3.5 Sonnet, and the 2026-era gpt-5.4-mini) as generators and decoders.
Datasets
The research relied on three primary corpora, split into "past" (for mapping) and "future" (for ground truth):
- Steam Games: ~110,000 game descriptions and metadata.
- arXiv CS Papers: ~113,000 titles and abstracts.
- US Patents: ~112,000 applications.
- Inspiration Pool: A diverse set of 49,531 items including Wikipedia articles, news, and recipes.
Key Insights and Ideas
- Far Points are Not Text: Random directions in high-dimensional embedding space are almost never "text." Even with sophisticated feedback, LLMs couldn't produce coherent descriptions that reached these far-flung coordinates.
- The Neighbor Trap: When an LLM is asked to "decode" a point by looking at its nearest neighbors, it doesn't find a new idea; it simply summarizes or blends those neighbors. As shown in Figure 1, the output is "causally anchored" to the examples it is shown.
- Momentum Beats Gaps: If you want to predict where the next 1,000 papers or games will appear, look at where they are already appearing. "Momentum" (recent growth) was a far stronger predictor of future density than "hole scores."
- The "Short Step" Problem: LLMs are inherently conservative. When asked to propose the "next" paper in a sequence, they consistently take a "shorter step" (staying closer to the source) than actual human researchers do.
Key Results
- Gap Scores Fail to Forecast: In tests on Steam and arXiv data, gap-based scores predicted future growth at or below chance levels (AUC 0.30 to 0.66). In contrast, momentum scores reached AUCs of 0.73 to 0.92.
- Orthogonality of Far Points: Coherent text constructed to target "interior gaps" reached a median cosine similarity of 0.57, but text targeting "far points" stayed near-orthogonal (median cosine 0.07).
- Feedback Helps, but Only So Much: Providing an LLM with distance-based feedback did help it take "longer steps" away from a source paper, but it didn't necessarily orient those steps toward the actual future research direction.
The overall results are summarized in Table 1, showing that most claims supporting gap-based ideation were "not supported" by the data.
Limitations and Future Work
The author notes several important caveats:
- Embedding Inversion: The study focused on neighbor-prompted decoders and searches. A "true" embedding inverter (like vec2text) might behave differently, though preliminary tests were uninformative.
- Inspiration Validation: The "Inspiration Distance" test failed its internal validation (the metric tracked format more than content), leaving the "Inverted U" hypothesis for AI ideation still open for debate.
- Encoder Specificity: Results are tied to the specific text encoders used (Qwen3 and BGE-M3). While these are representative, the "geometry of meaning" may shift with different architectures.
Future work could explore whether more sophisticated "agentic" workflows—where models iterate on ideas based on more than just distance scores—can overcome the conservative "short step" bias observed here.
Summary updated