Introduction
Traditional text categorization (TC) systems predominantly rely on the Bag of Words (BOW) model, which represents documents as simple frequency counts of their constituent terms. While computationally efficient, BOW models are fundamentally "brittle" because they lack external context. For instance, a model might fail to link "Bernanke" to the "Federal Reserve" or "Ciprofloxacin" to "antibiotics" unless these specific associations are explicitly present in the training data. This lack of world knowledge hinders performance on short documents and tasks involving synonymy or polysemy.
This research introduces a method to augment text classifiers with encyclopedic knowledge derived from Wikipedia. By mapping text fragments to a high-dimensional concept space, the proposed approach creates a "knowledge-rich" feature set that provides the context necessary for more human-like categorization.
Methodology: Wikipedia as a Concept Space
The core of the proposed system is a feature generation engine that utilizes Wikipedia as a structured proxy for human knowledge. The authors chose Wikipedia over other structured sources like the Open Directory Project (DMOZ) because Wikipedia articles are written in standard, "clean" English and possess a dense link structure that provides a richer web of conceptual relationships. The methodology involves several distinct phases:
- Concept Definition: Every Wikipedia article is treated as a distinct "concept." Using a 2005 snapshot, the authors curated a set of approximately 171,000 high-quality concepts by filtering out stubs and date-specific pages.
- Feature Generation: An auxiliary centroid classifier acts as a mapping function. Given a text segment, it identifies the top 10 most relevant Wikipedia concepts.
- Multi-Resolution Analysis: To address word sense disambiguation, the system analyzes text at multiple granularities—words, sentences, and paragraphs. For example, the term "Jaguar" is disambiguated by its proximity to words like "engine" (car) versus "jungle" (animal).
- Augmentation and Selection: The titles of relevant Wikipedia articles are appended as new features to the original BOW representation. Standard feature selection techniques are then applied to maintain a discriminative and manageable feature set.
Experimental Setup
The method was evaluated against five benchmark datasets: Reuters-21578, RCV1, OHSUMED (medical abstracts), 20 Newsgroups (20NG), and Movie Reviews. To test the system's robustness in data-sparse scenarios, the researchers also created "short document" variants of these datasets using only the document titles.
Results and Discussion
The Wikipedia-augmented approach consistently outperformed the BOW baseline across all datasets, demonstrating that providing a "cornucopia of world knowledge" can help machines compensate for limited reasoning skills.
Performance on Full Documents
As shown in Table 1, the integration of Wikipedia concepts led to substantial gains in both micro- and macro-averaged Break-Even Points (BEP). Notably, the macro-averaged performance improved by up to 30.4% on RCV1 and 18.0% on OHSUMED. These results suggest that external knowledge is particularly effective for categories with fewer training examples, where the model can leverage Wikipedia to bridge gaps in the training data.
The Short Document Advantage
The impact of encyclopedic knowledge was most pronounced in short documents. In scenarios where the BOW model has minimal text to analyze, Wikipedia features provide a critical context boost. As seen in Table 2, for certain OHSUMED categories, performance improved by over 35% when using only titles.
Conclusion
By augmenting traditional classifiers with Wikipedia concepts, this research effectively addresses the brittleness of BOW models. While the system introduces computational complexity due to the scale of the Wikipedia feature space, its ability to resolve ambiguity and handle short texts marks a significant step forward. Future work will explore deeper link analysis within Wikipedia and the application of this knowledge-rich representation to broader natural language processing tasks.
Summary updated