Do Embedding-Map Distances Guide LLM Ideation? Four Tests Against What Came Next

Javier Aguilar MartínarXiv 2026 · arXiv

The "gap-based ideation" recipe is a staple of AI-driven creativity hype: take a massive dataset (like all games on Steam), map them into an embedding space, find the "holes" where no data exists, and decode those coordinates into a brilliant new idea. It sounds mathematically sound, but does it actually work?

In a rigorous new paper, Javier Aguilar Martín puts this premise to the test. By using temporal splits—building maps from "past" data and checking them against "what came next"—the author finds that embedding distances are a poor guide for ideation. In short: the voids in our maps are usually empty for a reason, and they aren't hiding the next big thing.

TL;DR

This paper systematically debunks "gap-based ideation," the theory that "empty" spots in an embedding map represent novel, viable ideas. Through four pre-registered tests, the author shows that future work follows existing trends ("momentum") rather than filling gaps, and that LLMs struggle to generate coherent text that actually lands in these distant, empty regions.

What Problem Does This Paper Address?

As Large Language Models (LLMs) are increasingly used for scientific discovery and product design, researchers have sought ways to steer them toward "novelty." A popular approach is to use the geometry of embedding spaces—mathematical representations of meaning—to identify "content gaps."

The assumption is that distances in these maps carry information about where "good" new work lies. If we can find a coordinate far from existing work, we’ve found a "blue ocean" of opportunity. This paper asks if that assumption holds up when compared to actual historical data. Do the "holes" in a 2023 map of arXiv papers actually get filled by the breakthrough papers of 2024?

Methods

The author designed four tests to challenge the "gap" hypothesis:

  1. Decoding the Voids: Can an LLM actually write coherent text that corresponds to a "far" random point in vector space?
  2. Forecasting Growth: Do sparse regions of a map (gaps) grow faster in the future than dense regions?
  3. Steering Continuations: Can distance-based feedback help an LLM move "further" from a seed paper toward real-world follow-up research?
  4. Inspiration Distance: Is there an "optimal" distance for inspiration material (the "Inverted U" hypothesis)?

The study utilized a "past-only" map to ensure no data leakage and employed several state-of-the-art models (GPT-4o, Claude 3.5 Sonnet, and the 2026-era gpt-5.4-mini) as generators and decoders.

Datasets

The research relied on three primary corpora, split into "past" (for mapping) and "future" (for ground truth):

  • Steam Games: ~110,000 game descriptions and metadata.
  • arXiv CS Papers: ~113,000 titles and abstracts.
  • US Patents: ~112,000 applications.
  • Inspiration Pool: A diverse set of 49,531 items including Wikipedia articles, news, and recipes.

Key Insights and Ideas

  • Far Points are Not Text: Random directions in high-dimensional embedding space are almost never "text." Even with sophisticated feedback, LLMs couldn't produce coherent descriptions that reached these far-flung coordinates.
  • The Neighbor Trap: When an LLM is asked to "decode" a point by looking at its nearest neighbors, it doesn't find a new idea; it simply summarizes or blends those neighbors. As shown in Figure 1, the output is "causally anchored" to the examples it is shown.
  • Momentum Beats Gaps: If you want to predict where the next 1,000 papers or games will appear, look at where they are already appearing. "Momentum" (recent growth) was a far stronger predictor of future density than "hole scores."
  • The "Short Step" Problem: LLMs are inherently conservative. When asked to propose the "next" paper in a sequence, they consistently take a "shorter step" (staying closer to the source) than actual human researchers do.
Figure 1. Causal anchoring. Mean cosine of the decoded text to its target when the decoder is shown the target’s true neighbours, another target’s neighbours (swapped) or none, with the target and the score feedback unchanged; 20 targets per bar (luna = gpt-6-luna, Sonnet = Claude Sonnet 5.5). Swapping the neighbours lowers the landing on real and gap targets by 0.13 to 0.48 and removes it for far targets. The three panels have different y-axis scales (up to 0.8, 0.5 and 0.06).

Key Results

  • Gap Scores Fail to Forecast: In tests on Steam and arXiv data, gap-based scores predicted future growth at or below chance levels (AUC 0.30 to 0.66). In contrast, momentum scores reached AUCs of 0.73 to 0.92.
  • Orthogonality of Far Points: Coherent text constructed to target "interior gaps" reached a median cosine similarity of 0.57, but text targeting "far points" stayed near-orthogonal (median cosine 0.07).
  • Feedback Helps, but Only So Much: Providing an LLM with distance-based feedback did help it take "longer steps" away from a source paper, but it didn't necessarily orient those steps toward the actual future research direction.
Figure 2. Forecasting which past-map cells grow fastest. AUC for the top decile of growth, scored directly from each cell’s momentum, its registered hole score, or one valid gap score (H3, the share of uncovered probe points; raw space, from the validity study’s refit of the 256 cells), with bootstrap 95 % intervals; 26 positives per row. The registered hole score is shown in Section 4 to track cell position and size more than emptiness; the valid score, which detects planted holes, gives the same picture, and so do the other valid configurations (Table A3). The dashed line is chance.

The overall results are summarized in Table 1, showing that most claims supporting gap-based ideation were "not supported" by the data.

Table 1. Results at a glance. Basis: original = original registration, committed before its data; follow-up = registered follow-up, written after the main results were known; exploratory = not registered.

Limitations and Future Work

The author notes several important caveats:

  • Embedding Inversion: The study focused on neighbor-prompted decoders and searches. A "true" embedding inverter (like vec2text) might behave differently, though preliminary tests were uninformative.
  • Inspiration Validation: The "Inspiration Distance" test failed its internal validation (the metric tracked format more than content), leaving the "Inverted U" hypothesis for AI ideation still open for debate.
  • Encoder Specificity: Results are tied to the specific text encoders used (Qwen3 and BGE-M3). While these are representative, the "geometry of meaning" may shift with different architectures.

Future work could explore whether more sophisticated "agentic" workflows—where models iterate on ideas based on more than just distance scores—can overcome the conservative "short step" bias observed here.

Summary updated

Abstract

A popular ideation recipe treats an embedding map of existing work as a guide: pick points far from everything and decode them into ideas. I test four ways such distances could guide ideation against what appeared later. (1) Far random directions are not text: no later item reaches a raw cosine above about 0.10 with one (after centring or whitening, 22 of 24 cells stay out of reach), and an LLM decoder shown a target's neighbours, with score-only feedback, writes those neighbours in all six target sets. A neighbour-free search writes coherent text at median cosine 0.57 and 0.47 to interior gaps, beating the nearest existing item on 15 and 14 of 20, but only 0.07 and 0.06 to far points. (2) On k-means cells of a past-only map, the registered hole score mostly tracks cell size and position; three density-based gap scores forecast which cells grow fastest at or below chance (AUC 0.30 to 0.66) and add nothing to momentum in Steam and arXiv, with patents inconclusive; more isolated new Steam games are slightly better reviewed. (3) LLM continuations of arXiv papers stay closer to the source than papers that cite it, under three user prompts and against proposal rewrites (gpt-4o) and April-December 2023 papers (OLMo). Critique lengthens this step beyond content-free revision, replicated on 150 new papers. Revealing a real follow-up's title turns the ideas by 0.032 along a follow-up-specific direction beyond revision; reviewer, oracle and distance feedback add 0.13, 0.08 and 0.34 of that turn (upper bounds 0.46, 0.46, 0.70), and their turn is not separable from revision or their longer step. (4) For inspiration material graded by distance, no inverted U was detected, on an anticipation margin that tracks format more than content. Original tests were registered before running, follow-ups (thirteen requested in two AI-agent reviews) after the main results; all are dated only by local git commits.
BibTeX
@misc{martin2026do,
  author = {Javier Aguilar Martín},
  title = {{Do Embedding-Map Distances Guide LLM Ideation? Four Tests Against What Came Next}},
  year = {2026},
  eprint = {2611.21408},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21408},
}
Figure