Large language models (LLMs) are often celebrated for their emergent reasoning capabilities, yet their underlying mechanism remains a statistical engine trained to predict the next token. This creates a fundamental tension: when the logical requirements of a task conflict with the statistical distributions found in training data, models often abandon logic in favor of high-frequency patterns. This phenomenon, often termed word frequency bias, suggests that models frequently rely on lexical shortcuts rather than robust semantic or logical processing.
The Frequency Trap and Statistical Priors
At the core of this issue is the "frequency trap," where models default to predicting the most common words or entities they encountered during training. Frank et al. observed that standard models frequently default to the most common words within a document when faced with ambiguity, failing to select logically correct but less frequent terms. This behavior is not limited to general text; in specialized domains like medicine, Freitas et al. found that models often rely on the statistical prevalence of specific biological entities rather than learning the underlying relationships between them.
From a psychological perspective, Griffiths et al. argue that LLMs can be viewed as subjects with hidden inductive biases. When a task requires generating an output that is statistically rare, the model’s internal "priors"—the probability distributions learned from massive corpora—often override the deterministic rules of the prompt.
Positivity Bias and Scalar Blindness
One specific manifestation of frequency bias is positivity bias. Because positive-valence words (e.g., "more," "easier," "always") appear more frequently in natural language than their negative counterparts ("less," "harder," "never"), models develop a structural preference for them. Zhou et al. demonstrated that state-of-the-art models struggle with commonsense axioms when the correct answer requires a negative-valence word, defaulting instead to the more common positive term.
This bias extends to scalar adverbs. Lorge found that models like BERT and GPT-3 are often "blind" to the logical force of modifiers like perhaps or completely. Instead of processing the precise degree these adverbs imply, models are swayed by the raw frequency of the words, leading to predictions that are logically inappropriate for the context but statistically common in the training set.
The Probability Trap in Negation
Negation presents a unique challenge for frequency-driven models. In a "probability trap," the strong statistical association between two words can override an explicit negation marker like "not." For example, if the words "lion" and "ocean" are rarely associated, but "lion" and "savanna" are frequent, a model might struggle to process "A lion does not live in the..." because the associative strength of the high-frequency term overrides the logical instruction of the negation.
Gubelmann et al. argue that models are not necessarily "negation-blind" but are prone to a bias where high prior probabilities make it difficult to prune negated terms from their predictions. Similarly, Chen et al. identified a "belief conflict" where models can identify a statement as false in a quiz setting but will still hallucinate that same falsehood during open-ended generation because the positive association is statistically stronger than the negative one.
The Relative Frequency Heuristic
In Natural Language Inference (NLI), models often adopt a specific statistical shortcut known as the relative frequency heuristic. McKenna et al. discovered that models frequently assume entailment if the premise contains a rarer term than the hypothesis. This heuristic exploits a common linguistic pattern: specific, low-frequency words (e.g., "canary") often imply more general, high-frequency words (e.g., "bird").
However, this reliance on frequency distributions causes models to fail when the logical direction does not align with word rarity. Cheng et al. found that high-frequency predicates in a premise can override logical rules, especially when they conflict with the hypothesis. To combat this, the authors proposed using entailment graphs to generate counterfactual data, forcing models to prioritize structural logic over the statistical prevalence of predicates.
Key Foundational Work
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms (2021) — contributed 2 concept elements to this direction.
Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge (2018) — contributed 1 concept elements to this direction.