Introduction
In distributional semantics, thematic fit measures the plausibility of a noun filling a specific semantic role for a verb—for example, how well "hammer" fits as an instrument for the verb "hit" compared to "feather." Traditional unsupervised models evaluate this by creating a "prototype" vector, typically the average of the most frequent nouns appearing in that role. However, this "one-size-fits-all" centroid approach fails significantly when a verb is polysemous.
The Challenge of Polysemy
When a verb has multiple meanings, its typical role-fillers are distributionally diverse. Consider the verb serve and its "with"-prepositional objects. It may be used in culinary contexts (serve with salad), military contexts (serve with a regiment), or legal contexts (serve with a writ). Averaging these diverse vectors results in a prototype that represents a semantic "mixture" rather than any real-world usage, making it a poor benchmark for evaluating new candidate nouns.
Methodology: Multi-Prototype Clustering
The authors propose moving from a single centroid to a multi-prototype approach called kClusters. This method utilizes Agglomerative Clustering on the top $n$ typical role-fillers retrieved from a distributional model.
Distributional Models
The study evaluates three distinct vector spaces:
- TypeDM: A standard model based on syntactic dependencies.
- SDDM: A model based on semantic role labeling (SRL).
- SDDMX: A novel extension of SDDM that incorporates links between nouns belonging to the same predicate instance.
The kClusters Approach
Instead of a fixed number of clusters, the model uses the Variance Ratio Criterion (VRC) to dynamically determine the optimal number of clusters (ranging from 3 to 10). To evaluate a new noun, the model calculates its cosine similarity to every cluster centroid and selects the maximum score. This logic effectively asks if the noun fits any of the verb's known senses, rather than measuring its similarity to an average of all senses.
Evaluation and Results
The models were tested against four gold-standard human judgment datasets: Padó (2007), McRae et al. (1998), and two Ferretti et al. (2001) datasets focusing on instruments and locations. Performance was measured using Spearman’s $\rho$ to determine the correlation between model scores and human plausibility ratings.
As shown in the results, the kClusters method consistently outperformed the standard single Centroid and OneBest (nearest neighbor) baselines across nearly all datasets. In the aggregate evaluation, kClusters using the TypeDM model achieved a correlation of 0.385, compared to the 0.354 achieved by the centroid baseline.
Discussion
The success of the clustering approach highlights two key insights. First, Maximum Similarity is a more robust signal than average similarity for ambiguous words. Second, clustering acts as a natural noise filter; by isolating non-prototypical role-fillers into their own clusters, the model maintains cleaner prototypes for the primary senses.
The authors also noted that roles like locations and instruments benefited more from clustering than agents and patients, suggesting that the "where" and "how" of actions are often more distributionally diverse than the "who" or "what."
Summary updated