Abstract
Distributional semantic models typically require massive datasets to achieve high performance. However, in specialized domains such as philosophy, data is often a finite and scarce resource. This paper investigates the performance of various word embedding models—including Word2Vec, Singular Value Decomposition (SVD), and Nonce2Vec—on a "tiny" dataset consisting of the works of W.V.O. Quine. By benchmarking these models against a new, expert-curated conceptual network, we demonstrate that specialized architectures like Nonce2Vec and count-based SVD are more effective than standard predictive models for capturing domain-specific technical nuances, particularly for low-frequency terms.
Introduction
In Natural Language Processing (NLP), the prevailing paradigm suggests that "more data is better." Yet, for scholars in the humanities, the corpus of a single author may only span a few million words. This "small data" challenge is compounded by the technical depth of philosophical terminology, where words like "ontology" or "grounding" carry specific, author-dependent meanings. This study explores whether distributional models can be tuned to identify these subtle conceptual structures within limited corpora, specifically focusing on the English works of Willard Van Orman Quine.
Methodology
The research evaluates three primary approaches to generating word embeddings from a small corpus:
- Word2Vec (W2V): A standard predictive model tested with both random initialization and "warm-starting" using Wikipedia data.
- Singular Value Decomposition (SVD): A traditional count-based model known for its robustness to noise in smaller datasets.
- Nonce2Vec (N2V): A specialized variant of Word2Vec designed for incremental learning from minimal context using a high, rapidly decaying learning rate.
To address the lack of tuning data, the authors employed a "proxy" corpus of legal texts to optimize hyperparameters before applying the models to the philosophical domain.
Experimental Setup
The primary dataset, the QUINE Corpus, comprises approximately 2.15 million tokens representing the nearly complete English works of W.V.O. Quine. To evaluate the models, the authors introduced a novel Ground Truth benchmark: a conceptual network of 74 terms and 43 relations (e.g., reference, regimentation) categorized by experts into five thematic clusters: Language, Ontology, Reality, Mind, and Meta-linguistic. A 2019 Wikipedia dump served as a general-domain baseline.
Results and Analysis
The experimental results indicate that model architecture is more critical than data volume when dealing with technical, small-scale datasets.
As shown in Table 1, both Nonce2Vec (64.95%) and SVD (65.19%) significantly outperformed the in-domain Word2Vec (56.44%) and the Wikipedia-trained baseline (59.58%) in terms of cluster similarity. Furthermore, Nonce2Vec achieved a Dunn Index of 0.56, indicating much tighter and better-separated clusters compared to SVD (0.35) and Word2Vec (0.08).
Impact of Term Frequency
A critical finding was the models' performance relative to term frequency. Nonce2Vec, particularly with "additive" initialization (seeding a new word's vector with the average of its context), showed a clear advantage in handling rare terms.
The frequency analysis in Figure 1 reveals that while most models improve as term frequency increases, Nonce2Vec (N2V-add) maintains a higher performance baseline for low-frequency terms compared to standard Word2Vec variants. This makes it a valuable tool for philosophical research, where key technical terms may appear infrequently.
Discussion and Conclusion
While specialized models outperform general baselines, qualitative analysis suggests they are not yet "philosopher-ready." Models often struggled with technical polysemy—where terms have both common and technical meanings—and tended to collapse distinct expert categories into larger, less granular clusters.
Future work will focus on optimizing Nonce2Vec for "small" rather than "tiny" data and expanding the ground truth to include more balanced term sets. Additionally, the researchers aim to explore diachronic models to track how a philosopher’s conceptual definitions evolve over time, potentially providing new insights into intellectual history.
Summary updated