Spelling-out is not Straightforward: LLMs’ Capability of Tokenization from Token to Characters

Tatsuya Hiraoka, Kentaro Inui2025 · DOI

TL;DR

While Large Language Models (LLMs) can spell words perfectly, they don't actually "see" characters in their initial input. This paper reveals that character-level information is missing from the initial embedding layer and is instead dynamically reconstructed in middle-to-late Transformer layers at a specific "breakthrough" point.

What Problem Does This Paper Address?

There is a "spelling paradox" in modern AI. Most LLMs process text using subword tokens (like "lang" and "uage") rather than individual characters. Consequently, they often struggle with simple tasks like counting letters or identifying the fifth letter of a word, even though they can generate the sequence "l a n g u a g e" on command.

The authors investigate how and where this character-level knowledge is stored. If the model doesn't have a "character map" in its initial vocabulary, how does it manage to spell? This research aims to uncover the internal machinery that allows a token-based model to perform character-level operations.

Methods

The researchers employed a "mechanistic interpretability" approach, using four medium-sized models (LLaMA3-8B, Gemma-7B, Qwen2.5-7B, and Amber-6.7B) as subjects. Their toolkit included:

  • MLP Probing: They trained small diagnostic classifiers to see if they could "read" the identity of the $N$-th character from the model's internal hidden states at various layers.
  • Knowledge Neuron Identification: Using integrated gradients (an attribution method), they pinpointed specific neurons in the Feed-Forward Networks (FFN) that contribute most to predicting the correct character.
  • Neuron Ablation: They "turned off" these identified neurons to see if the model's spelling ability broke, confirming the neurons' importance.
  • Attention Inspection: They analyzed where the model "looks" (attention weights) while it is in the process of spelling out a word.

Datasets

The authors constructed a new token-characters dataset derived from the actual vocabularies of the four target LLMs.

  • Criteria: They selected single tokens that consist only of lowercase English letters (a-z), start with a word-head prefix (like the " " in LLaMA), and are at least five characters long.
  • Scale: This resulted in datasets ranging from ~6,000 to ~47,000 tokens depending on the model's vocabulary size.
  • Novelty: Unlike previous benchmarks that use full words, this focuses on the model's internal "atoms" (tokens), ensuring the model isn't just relying on external context to guess the spelling.

Key Insights and Ideas

  • The Embedding Blind Spot: The initial embedding layer (Layer 0) is surprisingly ignorant. It can identify the first character of a token with high accuracy, but its knowledge of the second, third, and subsequent characters is nearly non-existent.
  • The "Breakthrough" Layer: Character knowledge doesn't emerge gradually. Instead, there is a distinct "breakthrough" point in the middle layers where the model's internal representations suddenly become highly predictive of character identity.
  • Dynamic Reconstruction: Spelling isn't a retrieval task; it's a reconstruction task. The model uses its early layers to identify the token and then uses its middle-to-late layers to "unpack" that token into its constituent characters.
  • General vs. Specialized Neurons: The model uses a broad, shared set of neurons to handle the first character of any word. However, as it moves deeper into the word (the 3rd or 4th character), it switches to more specialized, position-specific neurons.

Key Results

  • Spelling Accuracy: LLaMA3-8B was the top performer, spelling tokens correctly 94.41% of the time, while Amber-6.7B struggled at 58.86%.
  • Probing Jump: In LLaMA3-8B, probing accuracy for the 5th character jumps from ~40% in early layers to nearly 100% after the breakthrough point (around layer 16).
  • Attention Correlation: The layer with the highest attention weight on the target token almost perfectly aligns with the "breakthrough" layer where character information becomes detectable.
  • Ablation Impact: Removing the top 100 "spelling neurons" for the first character had a negligible effect, but removing them for the 3rd character or beyond caused a significant drop in accuracy, proving that later-stage spelling is more dependent on specific internal "specialists."

Limitations and Future Work

  • Language & Script: The study is limited to lowercase English alphabets. The authors acknowledge that different patterns might emerge in morphologically rich languages or non-alphabetic scripts (like Chinese radicals).
  • Model Size: The study focused on ~7B parameter models. It remains to be seen if massive models (like GPT-4) or tiny edge models follow the same "breakthrough" logic.
  • Amber as an Outlier: One model (Amber-6.7B) did not follow the same patterns as the others, suggesting that pre-training data and specific architectural choices can fundamentally change how a model stores character data.
  • Future Direction: The authors suggest this framework could be used to improve LLM reliability in tasks like handling typos or performing complex string manipulations.

Summary updated

Abstract

Large language models (LLMs) can spell out tokens character by character with high accuracy, yet they struggle with more complex characterlevel tasks, such as identifying compositional subcomponents within tokens.In this work, we investigate how LLMs internally represent and utilize character-level information during the spelling-out process.Our analysis reveals that, although spelling out is a simple task for humans, it is not handled in a straightforward manner by LLMs.Specifically, we show that the embedding layer does not fully encode character-level information, particularly beyond the first character.As a result, LLMs rely on intermediate and higher Transformer layers to reconstruct character-level knowledge, where we observe a distinct "breakthrough" in their spelling behavior.We validate this mechanism through three complementary analyses: probing classifiers, identification of knowledge neurons, and inspection of attention weights.
BibTeX
@misc{hiraoka2025spelling,
  author = {Tatsuya Hiraoka and Kentaro Inui},
  title = {{Spelling-out is not Straightforward: LLMs’ Capability of Tokenization from Token to Characters}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.719},
}
Figure