Abstract
This paper presents a comparative study of seven machine learning algorithms applied to the task of Word Sense Disambiguation (WSD). By evaluating statistical, neural, and symbolic paradigms on a standardized dataset, the research identifies that Naive Bayes and Perceptron models significantly outperform rule-based and decision tree methods. The findings highlight the importance of "inductive bias," suggesting that WSD is best addressed by models that integrate many weak indicators rather than those seeking a compact set of rules.
Introduction
In the mid-1990s, the field of Natural Language Processing (NLP) lacked rigorous head-to-head comparisons between emerging learning paradigms. This study addresses this gap by evaluating how different algorithmic assumptions—or inductive biases—affect performance in determining the correct meaning of polysemous words. The central hypothesis is that the success of an algorithm depends on how well its bias matches the inherent structure of the problem.
Experimental Setup
The experiments utilized a preprocessed corpus focused on the word "line," which has six distinct meanings (e.g., text, phone, product). The dataset consisted of 2,094 examples, each represented by 2,859 binary features indicating the presence of word stems in the local context.
Seven algorithms were compared:
- Statistical: Naive Bayes
- Neural: Perceptron
- Decision Trees: C4.5
- Instance-Based: 3-Nearest Neighbor
- Logic-Based: Three variants of FOIL (PFOIL-DNF, PFOIL-CNF, PFOIL-DLIST)
Performance was measured using learning curves, tracking accuracy as training data increased from 0 to 1,200 examples.
Results and Analysis
The experimental results revealed a clear performance hierarchy. Naive Bayes and the Perceptron consistently achieved the highest accuracy, reaching approximately 70-72% correct disambiguations at the 1,200-example mark. These results were statistically significant ($p \le 0.05$) compared to all other methods.
The Role of Inductive Bias
The superior performance of statistical and neural methods is attributed to their inductive bias. WSD typically lacks a "smoking gun" feature; instead, it relies on "many weak indicators." Naive Bayes and Perceptrons are biased toward using a weighted combination of all available features. In contrast, symbolic methods like C4.5 and PFOIL are biased toward compactness, attempting to find a small subset of highly predictive features. This "compactness bias" proves detrimental in the context of broad linguistic evidence.
Efficiency Trade-offs
While statistical models provided higher accuracy, they exhibited a significant trade-off in computational efficiency during inference. As shown in Figure 3, symbolic methods (C4.5 and PFOIL) were exceptionally fast at testing, requiring minimal CPU time. Naive Bayes, despite being fast to train, was the slowest during the testing phase due to the overhead of processing thousands of features per instance.
Conclusion
The study demonstrates that for tasks like Word Sense Disambiguation, algorithms that aggregate broad contextual evidence outperform those that prioritize rule simplicity. While symbolic methods offer superior inference speed and human-readability, their inductive bias is poorly suited for the "many weak indicators" nature of human language. Future work should investigate if richer linguistic representations (e.g., part-of-speech tags or syntax) might alter these performance dynamics.
Summary updated