Abstract
This paper introduces the Semantic Pleonasm Corpus (SPC), the first dataset specifically designed to help AI detect stylistic redundancies such as "free gift" or "plain pure." By moving beyond functional grammar errors, this work provides a benchmark for systems to improve writing conciseness. The study demonstrates that while identifying if a sentence is redundant is relatively straightforward (92.2% accuracy), pinpointing the exact word to remove remains a significant challenge for current NLP models, with the best ensemble achieving only 39.4% accuracy.
Introduction
In writing, a pleonasm is the use of more words than necessary to convey a meaning—common examples include "tuna fish," "ATM machine," or "advance warning." While these are not typically "wrong" in a grammatical sense, they clutter prose and increase cognitive load for readers and complexity for Natural Language Processing (NLP) tasks like machine translation or summarization.
Existing Grammar Error Correction (GEC) datasets primarily focus on functional errors (e.g., preposition misuse). They lack examples of semantic redundancy, where words are grammatically sound but rhetorically redundant. Furthermore, existing models often rely on simple heuristics like semantic distance; however, many valid expressions use closely related words for nuance, making simple distance a poor metric for true pleonasm detection.
The Semantic Pleonasm Corpus (SPC)
The authors introduced the Semantic Pleonasm Corpus (SPC), containing 3,019 sentences annotated for redundancy. Data was sourced from amateur restaurant reviews (Yelp), which frequently contain redundant descriptors.
The corpus construction involved a two-step process:
- Heuristic Filtering: Adjacent word pairs with high semantic similarity were identified using WordNet and TextBlob.
- Human Annotation: Amazon Mechanical Turkers reviewed candidates to decide if the first word, second word, both, or neither were redundant.
Crucially, SPC includes "near-miss" negative examples—sentences with semantically related words that are not redundant—to prevent models from over-relying on simple similarity metrics.
Methodology
The research evaluated two primary tasks: word-level identification and sentence-level detection.
Word-Level Identification
Four metrics were tested to identify specific redundant words:
- Xue & Hwa (2014): A baseline using language modeling and translation alignments.
- SIM: Measures semantic similarity between the full sentence and a version with the target word removed.
- GEN: Measures word "generality" based on synonym counts.
- SMP: Measures word simplicity using readability scores.
Sentence-Level Classification
Binary classifiers (Naive Bayes and Maximum Entropy) were built to determine if a sentence contains any pleonasm. Features included unigrams (UG), trigrams (TG), TF-IDF, and word-level redundancy statistics (WSTAT).
Key Results
Word-Level Performance
On the SPC dataset, the best combined model (ALL) achieved 39.4% accuracy in identifying the redundant word. While this highlights the difficulty of the task, it significantly outperformed the GEC-based baseline (4.7%), proving that semantic pleonasm requires a specialized approach distinct from standard grammar checking.
Sentence-Level Performance
The binary classifier achieved high performance, with the Naive Bayes model reaching 92.2% accuracy when combining TF-IDF features with word-level statistics (WSTAT). This suggests that models are quite effective at flagging "wordy" sentences even if they struggle to pinpoint the exact redundant token.
Discussion and Future Work
The results suggest a massive gap in difficulty between detecting a problem and fixing it. No single metric (similarity, generality, or simplicity) was sufficient on its own; combining these different perspectives led to the best performance.
Future work should address the current adjacency constraint, as redundancies often occur across a sentence (e.g., "The reason why... is because"). Additionally, moving from detection to automated correction—where the system suggests a more concise rewrite—remains the ultimate goal for improving writing assistance tools.
Summary updated