LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems

Nan Xu, Xuezhe Ma

Abstract

Large Language Models (LLMs) demonstrate expert-level proficiency in complex mathematics and coding yet frequently fail at trivial tasks such as counting character occurrences in simple words. This study investigates the "Genius Paradox," evaluating common hypotheses including subword tokenization limitations and lack of character-level training. Through a series of perturbation analyses and reasoning elicitation experiments, we demonstrate that the failure is not a lack of fundamental capability but a failure to trigger appropriate reasoning processes. We find that while tokenization is often blamed, forcing character-level input does not consistently resolve the issue. Conversely, prompting for explicit reasoning steps allows models like GPT-4o to achieve near-perfect accuracy.

Introduction

The research community has been puzzled by a glaring inconsistency: LLMs exhibit expert-level performance in coding and complex mathematics yet stumble over trivial, word-based counting problems that a primary school student could solve. This "Genius Paradox" challenges our understanding of how these models perceive and process language.

The authors investigate three prevalent conjectures for this failure:

  1. Tokenization Issues: Subword tokenization "hides" individual letters from the model.
  2. Lack of Character-level Training: Models haven't seen enough character-level data during pretraining to understand internal word structures.
  3. Model Size/Architecture Constraints: Embedding size or transformer architecture inherently limits character counting.

Methods

The study employs a rigorous evaluation framework across four specific tasks: counting character occurrences (Task I), identifying substrings (Task II), determining word length (Task III), and counting unique characters (Task IV).

To test the conjectures, the researchers used:

  • Perturbation Analysis: Modifying inputs (e.g., s-t-r-a-w-b-e-r-r-y) to force character-level visibility.
  • Cross-Domain Reasoning: Testing sentiment analysis on text formatted entirely as individual characters to check for latent perception.
  • Reasoning Elicitation: Applying strategies like Chain-of-Thought (CoT), Self-Consistency, and Tree-of-Thought (ToT).

The benchmark consists of 500 words sampled from the NLTK library, alongside standard datasets like IMDB and SST-2 for sentiment analysis.

Key Insights

The Tokenization Myth

A common excuse for LLM counting failures is that subword tokenization prevents the model from "seeing" individual letters. However, experimental results refute this. Forcing the model to process individual characters through explicit character-level tokenization or input perturbations (like adding dashes) did not consistently improve performance. In many cases, performance actually degraded when models were forced into character-level processing.

Accuracy comparison between original subword tokenization and forced character-level input across various models.

Latent Capability

The study reveals that the capability to perceive characters is "latent." Models performed exceptionally well at sentiment analysis on character-spaced text, proving they can reason with character-level information when the task demands it. The failure in counting tasks is therefore not a lack of perception, but a failure to apply that perception to the specific problem of counting.

Comparison of model performance on word-level vs. character-level inputs, demonstrating latent character perception capabilities.

The Role of Reasoning

The most significant finding is that reasoning is the "silver bullet." While base models struggle, adding a reasoning step like Chain-of-Thought (CoT) dramatically boosts accuracy. GPT-4o, for instance, reached 100% accuracy across all counting tasks when using reasoning strategies. Self-Consistency emerged as the most robust method for improving performance across different model families.

Conclusion

The "Genius Paradox" is not a fundamental architectural limitation but a behavioral one. LLMs possess the latent ability to perceive and reason about individual characters, but they often fail to trigger these processes for simple counting tasks. While inference-time reasoning strategies like CoT provide a temporary fix, the long-term solution likely involves cultivating "reasoning before responding" during the pretraining phase.

Summary updated

BibTeX
@inproceedings{xu2025llm,
  author = {Nan Xu and Xuezhe Ma},
  title = {{LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with Simple Word-based Counting Problems}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.172},
}
Figure