Dancing Tokens on the Vocabulary Network from LLM Generation

Yuan-Chao HuarXiv 2026 · arXiv

TL;DR

Researchers have developed a way to visualize Large Language Model (LLM) behavior by mapping text generation onto a "vocabulary network" built from token embeddings. Their findings reveal that generation is not a simple walk between similar words, but a complex leap across a fragmented, scale-free graph—a perspective that could help detect and prevent model "degeneration" (repetitive loops) in real-time.

What Problem Does This Paper Address?

Despite their ubiquity, LLMs remain "black boxes." We understand the math of the Transformer architecture, but we lack a "microscopic" understanding of how a model navigates its vast vocabulary during inference. Specifically, it is unclear how the static geometry learned during pre-training (the embeddings) relates to the dynamic process of generating text.

Existing explainability methods often treat the model as a whole or look at internal activations, but they rarely reference the actual "space" where tokens live. This paper seeks to bridge that gap by treating the vocabulary as a physical network and observing how the model "dances" across it.

Methods

The authors constructed a token-similarity network for six open-source LLMs.

  1. Network Construction: Every token in the vocabulary is a node. An edge is drawn between two nodes if the cosine similarity of their input embeddings exceeds a threshold $S$.
  2. Reference Cutoff: To make models comparable, they set a threshold $S_c$ where the mean degree of the network is exactly 1 (the point where a "giant component" typically begins to emerge in random graphs).
  3. Logit Lens: They used the "logit lens" technique to project the hidden states of every internal layer back into the vocabulary space. This allowed them to identify the "settling layer"—the earliest point in the model's depth where it commits to the final output token.
  4. Mapping Generation: By treating the generated sequence as a series of directed steps on this static graph, they could visualize whether the model stays in a "neighborhood" or jumps between distant clusters.
Figure 1: Schematic of autoregressive generation mapped onto a token graph. a, The tokens of an input sequence are located as nodes of the token-similarity network (grey edges), and their order is drawn as directed sequence steps (blue arrows); sequence steps need not coincide with similarity edges. b, The predicted next token is located as a further node, joined to the sequence by a new step (orange; dashed, predicted step). c, The extended sequence becomes the new input (orange step now part of the sequence) and the following token is predicted (green, dashed). d, Iterating a→c maps the full generation onto the network (one colour per generated step). Linked squares beneath each panel show the corresponding input (blue) and generated (coloured) token sequence.

Datasets

The study analyzed six prominent open-source models:

  • Meta-Llama-3.1-8B (Pre-trained base)
  • Qwen3-8B (Post-trained)
  • DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Qwen-7B (Distilled reasoning models)
  • DeepSeek-R1-0528-Qwen3-8B
  • gpt-oss-20B (Mixture-of-Experts reasoning model)

Evaluation was performed using nine prompts across three distinct domains: metallic glasses (materials science), machine learning, and movies.

Key Insights and Ideas

  • The Vocabulary is an Archipelago: Unlike random embeddings, trained LLM embeddings form a highly fragmented network. It is "scale-free" (a few hubs have many connections, most have few), but these hubs are often punctuation or formatting tokens rather than semantic concepts.
  • Generation is Not a Walk: Surprisingly, consecutive tokens in a sentence are rarely neighbors in the similarity network. In fact, they are often less likely to be neighbors than tokens chosen at random from the same text. This implies that the attention mechanism is constantly pulling the model across different "islands" of the vocabulary.
  • Distillation is an Inheritance: Distilling a model (like DeepSeek-R1 into Llama) drastically changes its reasoning capabilities but barely moves its input embeddings. The "knowledge space" is largely fixed during the initial pre-training phase.
  • Degeneration as Confinement: When a model starts repeating itself (hallucinating or looping), it physically collapses into a small, densely connected sub-graph. This "spatial collapse" can be measured, offering a new way to monitor model health during inference.
The degree distribution of the token networks follows a heavy-tailed power law, a hallmark of scale-free networks.

Key Results

  • Power-Law Topology: All models exhibited a degree distribution $P(k) \sim k^{-\gamma}$ with $\gamma \approx 2.19$, similar to word co-occurrence networks in natural language.
  • Late Settling: In most models, the final token identity only "settles" very late in the model's depth (median depth $\sim 0.87$ of the total layers). Base models settle earlier than reasoning-tuned models.
  • Domain Specificity: 70% of tokens generated for a specific domain (e.g., metallic glasses) are never used in other domains. This suggests that models could potentially be "pruned" into domain-specific versions by identifying these functional sub-graphs.
  • Repetition Metrics: The authors found a strong negative correlation ($r = -0.98$) between the "token-revisit rate" on the graph and the diversity of the generated text, providing a quantitative metric for "trapped" generation.

Limitations and Future Work

The authors acknowledge that their current network is built on static input embeddings. In reality, Transformers use dynamic attention that changes the "effective" similarity between tokens based on context. Future work aims to incorporate these context-dependent attention scores into the graph.

Additionally, the study suggests that "strategic path optimization" on these graphs could lead to better decoding algorithms. Instead of just picking the most likely next token, we might steer the model toward under-explored but promising regions of the vocabulary network to improve creativity and reduce hallucination.

Summary updated

Abstract

Large language models (LLMs) generate text one token at a time, yet how a particular generation relates to the geometry learned in pre-training remains unclear. The fundamental explainability of LLMs is elusive. Here, we build a similarity network from the input embeddings of six open LLMs and map generation onto it. The token-embedding network is scale-free while heavy-tailed, as are random embeddings with matched covariance, but far more fragmented. Distillation barely changes this topology because input embeddings are largely inherited. The knowledge space is thus deterministic from pre-training. Consecutive generated tokens are rarely graph neighbours and, in most models, no more often than in shuffled order, so generation is not a walk on the predefined network, although the tokens it visits are concentrated within it. Through the logit lens in a layer- wise manner, tokens settle late in depth in most models, and repetitive models remain confined to small domain-specific sub-graphs. These observables could serve as inference-time monitors of degeneration. We thus provide a fresh perspective on understanding LLM generation in the microscopic way from node or link prediction by graph learning.
BibTeX
@misc{hu2026dancing,
  author = {Yuan-Chao Hu},
  title = {{Dancing Tokens on the Vocabulary Network from LLM Generation}},
  year = {2026},
  eprint = {2611.18248},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18248},
}
Figure