Introduction
Discovering phrasal synonyms is a critical task for natural language processing, yet it remains challenging due to the variability of multi-word expressions. This paper introduces NeSS (Near-Synonym System), an unsupervised framework designed to identify phrasal synonyms from large-scale monolingual corpora. Unlike traditional methods that depend on bilingual parallel data or complex linguistic rules, NeSS relies on statistical distributions to find semantically equivalent phrases, even when they share no common words.
The Challenge of Phrasal Synonymy
While single-word synonymy is well-documented in thesauri, phrasal synonymy is more elusive. Phrases like "cheap housing" and "affordable homes" are semantically identical but lack lexical overlap. Existing approaches often suffer from three main limitations:
- Bilingual Pivoting: Dependency on massive parallel corpora (e.g., English-French) to bridge phrases.
- Linguistic Dependency: Reliance on language-specific tools like parsers, which limits portability.
- Static Nature: Precompiled lists fail to capture the evolving "long tail" of natural language.
NeSS addresses these by providing a "live," scalable system that generates synonyms on-the-fly using raw, unannotated text.
The NeSS Framework
The core of NeSS is an extension of Harris’ Distributional Hypothesis: phrases appearing in similar contexts likely share similar meanings. The system follows a multi-stage pipeline to extract and rank candidates.
The Pipeline Stages:
- Context Collection: For a query phrase (e.g., "sea change"), the system identifies its occurrences and extracts surrounding contexts (left, right, and "cradle").
- Candidate Generation: The corpus is searched for other phrases appearing in these identical contexts.
- Scoring (Shared Feature Gain): Candidates are ranked based on a probabilistic model incorporating Contextual Query Phrase Relevance (CQR), Candidate Contextual Strength (CCS), and Contextual Information (Inf).
- Re-ranking (KL Divergence): The top 1000 candidates are refined using Kullback-Leibler divergence to compare the full contextual distributions of the query and the candidate.
Engineering for Scale
To process billions of words in real-time, NeSS utilizes Suffix Arrays, enabling $O(\log N)$ search complexity. By partitioning the corpus across multiple threads, the system achieves performance roughly 200x faster than previous Hadoop-based state-of-the-art methods.
Experimental Results
The system was evaluated using the English Gigaword Fifth Edition (~1.51 billion words). Performance was measured using the **$MR\$(n)$ (Mean Rank-Sensitive score)**, which weights the quality of the top $n$ results based on human judgments.
NeSS significantly outperformed existing baselines, including the Paraphrase Database (PPDB) and traditional thesauri. For two-word phrases, NeSS achieved an $MR\$(5)$ of 2.45, compared to 1.97 for PPDB. Notably, for phrases longer than two words, NeSS maintained high performance (2.15) while thesaurus-based methods dropped significantly (0.50). It also surpassed neural word embedding models in single-word synonym extraction, highlighting the precision of distributional n-gram methods.
Key Insights and Future Work
The success of NeSS underscores that context specificity is paramount; longer, more specific shared contexts are stronger indicators of synonymy. Furthermore, the system's language-independent nature makes it a versatile tool for any language with a sufficiently large corpus.
Future work includes addressing data sparsity for rare phrases and testing the system on non-English languages to further validate its language-agnostic design. Integrating NeSS into downstream tasks like query expansion for web search remains a promising direction for real-world application.
Summary updated