Acquisition of Semantic Classes for Adjectives from Distributional Evidence

Gemma Boleda, Toni Badia, Eloi BatlleCOLING 2004

TL;DR

This paper explores how to automatically group adjectives into semantic categories (like "basic," "event," or "object" properties) using only their usage patterns in text. They demonstrate that shallow syntactic features are surprisingly effective for this task, achieving results comparable to human agreement, and that this syntactic approach is more robust than relying on morphology alone.

What Problem Does This Paper Address?

The field of Natural Language Processing (NLP) systems often requires a rich understanding of word meanings, particularly for tasks like question answering or information extraction. However, manually creating comprehensive lexicons of word properties is incredibly time-consuming and expensive. While significant research has focused on acquiring semantic classes for verbs, adjectives have received less attention. This paper aims to bridge this gap by developing an automatic method to classify adjectives into meaningful semantic groups, specifically for the Catalan language, which has fewer linguistic resources than English. The core challenge is to infer these semantic properties from how words are used in large text corpora, a concept known as distributional semantics.

Methods

The authors employ a clustering experiment, an unsupervised machine learning technique. This means they don't pre-define the categories but rather let the algorithm discover them based on patterns in the data. They represent adjectives using shallow distributional features, specifically focusing on bigrams (pairs of adjacent words) within a four-word window around the adjective. These features capture local syntactic context.

They define a classification scheme for adjectives based on two parameters:

  1. Unary vs. Binary: Whether an adjective denotes a single property (e.g., "red") or a relation between two entities (e.g., "jealous of").
  2. Basic, Object, or Event Denotation: Whether the adjective denotes a fundamental property (e.g., "happy"), has an embedded object component (e.g., "pulmonary" disease), or denotes a state dependent on an event (e.g., "constitutive").

They then use the k-means clustering algorithm with specific feature normalization techniques to group the adjectives. To evaluate their approach, they create a Gold Standard by having human judges manually classify a subset of adjectives according to the defined parameters.

Datasets

  • Catalan Corpus: A 16.5 million word corpus of modern written Catalan, semi-automatically tagged and hand-corrected.
  • Tuning Subset: A manually classified subset of 100 adjectives used for feature selection and parameter tuning.
  • Testing Subset (Gold Standard): A manually classified subset of 80 adjectives used as the benchmark to compare the clustering results against human judgment.

Key Insights and Ideas

  • Syntax Trumps Morphology for Semantic Classification: A key finding is that syntactic distribution (how words are used in sentences) is a more reliable indicator of an adjective's semantic class than its morphological origin (e.g., whether it's derived from a noun or verb). This is particularly true for adjectives that have undergone semantic shifts over time.
  • Shallow Features are Powerful: Surprisingly, simple bigram features capturing local syntactic context are sufficient to achieve a broad semantic classification of adjectives. This is a significant finding, as it suggests that complex linguistic analysis is not always necessary.
  • Event Adjectives are Tricky: The "event" adjective class proved to be the most challenging to cluster accurately. The clustering tended to group binary event adjectives with other binary adjectives, suggesting that the definition or syntactic behavior of event adjectives might be less homogeneous than initially assumed.
  • Human Agreement is High but Not Perfect: The human judges showed remarkably high agreement on the classification task, especially for the unary/binary parameter. This validates the classification scheme itself and suggests that the task, while requiring expertise, is indeed feasible for humans.

Key Results

  • Unary/Binary Classification: The clustering achieved an agreement score of 0.97% and a kappa statistic of 0.87 with the Gold Standard. This is highly comparable to inter-judge agreement and indicates that the clustering effectively distinguishes between unary and binary adjectives.
  • Basic/Event/Object Classification: The agreement for this parameter was lower, with 0.73% and a kappa of 0.56. While still showing a reasonable correspondence, it highlights the greater difficulty in distinguishing these semantic types.
  • Morphology vs. Syntax Comparison: The agreement between a morphology-based classification and the Gold Standard was significantly lower (0.65% and kappa 0.49) than the syntax-based clustering, supporting the paper's hypothesis.

Limitations and Future Work

  • Event Adjective Definition: The authors acknowledge that the "event" adjective class is not well-identified by the clustering. Future work will involve reviewing and refining the definition and characterization of this class.
  • Gold Standard Size: The current Gold Standard is based on a relatively small sample of manually annotated adjectives. The authors plan to create a larger Gold Standard to establish statistically more reliable results and conduct further analyses, such as examining how clustering errors relate to inter-judge disagreement.
  • Alternative Models: Further experiments with alternative modeling strategies and clustering algorithms are planned to perform a more global analysis of the approach.
  • Polysemy: The issue of polysemy (words having multiple meanings) was not fully addressed due to low agreement among judges. Defining and integrating polysemy into the task is a crucial next step.
  • Cross-Lingual Extension: The authors believe their approach could be extended to other Indo-European languages, but this remains future work.

Summary updated

BibTeX
@inproceedings{boleda2004acquisition,
  author = {Gemma Boleda and Toni Badia and Eloi Batlle},
  title = {{Acquisition of Semantic Classes for Adjectives from Distributional Evidence}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2004},
}
Figure