Abstract
Human readers possess a remarkable ability to process jumbled text, provided the first and last letters of words remain in place—a phenomenon known as the "Cambridge University" effect. In contrast, modern Natural Language Processing (NLP) systems are often fragile when encountering even minor character-level noise. This paper introduces the semi-character Recurrent Neural Network (scRNN), a cognitively-inspired architecture that represents words as a combination of their first character, last character, and an unordered bag of internal characters. Our results demonstrate that the scRNN achieves nearly 99% accuracy in correcting jumbled words, significantly outperforming traditional spell checkers and character-level deep learning models.
Introduction
The robustness of human word recognition stands in stark contrast to the sensitivity of state-of-the-art NLP models. While humans can effortlessly read "Aoccdrnig to a rscheearch," a single typo can derail a character-level CNN or a standard LSTM. Traditional error correction methods often rely on edit distances, which do not account for the position-dependent nature of human perception. We propose a model that is "cognitively plausible," leveraging the psycholinguistic insight that the first and last letters of a word serve as critical anchors for recognition.
Methodology: The scRNN Architecture
The core innovation of this work is the semi-character Recurrent Neural Network (scRNN). Unlike standard models that process character sequences linearly, the scRNN employs a word-level representation that is invariant to internal permutations.
Each word is converted into a single input vector $x_n$ consisting of three distinct components:
- Beginning ($b$): A one-hot encoding of the first character.
- Internal ($i$): A "bag of characters" (count vector) representing all characters between the first and last, ignoring their specific order.
- Ending ($e$): A one-hot encoding of the last character.
This input is processed by a Long Short-Term Memory (LSTM) network, which predicts the correct word from a fixed vocabulary. The model is remarkably efficient; while larger configurations were tested, a version with just 50 hidden units (approximately 2MB) proved sufficient for high-accuracy recognition.
Experimental Setup
The model was trained and evaluated using the Penn Treebank (PTB) corpus. To test robustness, the researchers introduced three types of synthetic noise:
- Jumble: Shuffling internal characters (e.g., Cambridge $\rightarrow$ Cmabrigde).
- Delete: Removing one internal character.
- Insert: Adding one extra internal character.
Results
The scRNN demonstrated exceptional performance, particularly on jumbled text. While a standard CharCNN achieved only 17.17% accuracy on jumbled words, the scRNN reached 98.96%. It also outperformed commercial spell checkers, which typically hovered around 55% accuracy.
As shown in the accuracy plots above, the model converges rapidly across various hyperparameter configurations ($\beta$ values). Beyond jumbling, the model showed surprising generalization to deletions (85.74%) and insertions (96.70%). Furthermore, the RNN structure allows the model to use sentence context to disambiguate anagrams (e.g., deciding between "mass" and "miss" based on surrounding words).
Cognitive Alignment and Discussion
A key finding is that the scRNN's performance degradation mirrors human difficulty. Psycholinguistic studies show that jumbling the beginning or ending letters of a word significantly increases human eye fixation time compared to internal jumbling. When the researchers modified the scRNN to jumble "anchor" positions, its accuracy dropped sharply, aligning the model's failure modes with human cognitive constraints.
Despite its success, the model has limitations. The "bag of characters" approach makes it susceptible to internal anagrams (e.g., "there" vs. "three") in ambiguous contexts. Additionally, the current implementation relies on a fixed vocabulary, limiting its ability to handle out-of-vocabulary (OOV) terms. Future work will focus on extending this architecture to handle more naturalistic noise, such as OCR errors and social media slang.
Summary updated