Automatic Extraction of Turkish Hypernym-Hyponym Pairs From Large Corpus

Savaş Yildirim, Tuğba YildizCOLING 2012

TL;DR

This paper presents a fully automatic system for extracting hypernym-hyponym (is-a) relationships from Turkish text. By combining lexico-syntactic patterns with a novel bootstrapping approach for semantic expansion, the system achieves high precision and recall, offering a valuable resource for Turkish NLP tasks.

What Problem Does This Paper Address?

Building comprehensive semantic lexicons, like WordNet, is crucial for many Natural Language Processing (NLP) tasks. However, creating these resources is labor-intensive and time-consuming, especially for languages with fewer existing resources. While methods for extracting hypernym-hyponym pairs from text exist, they often rely on hand-crafted patterns or require significant manual effort. This paper addresses the challenge of automatically acquiring these "is-a" relationships from a large Turkish corpus, aiming for both high precision and recall without human intervention.

Methods

The proposed system employs a two-stage approach:

  1. Initial Seed Extraction using Lexico-Syntactic Patterns: The system first leverages a set of predefined lexico-syntactic patterns specifically designed for Turkish to identify potential hypernym-hyponym pairs. These patterns capture common linguistic constructions that indicate an "is-a" relationship. To improve the reliability of these initial extractions, several elimination rules are applied to filter out incorrect pairs. These rules include checking for nominative case, requiring multiple occurrences of a pattern match, and ensuring the hyponym is not more general than the hypernym (based on frequency).
  2. Statistical Expansion via Bootstrapping: To overcome the limitations of pattern-based extraction (which can lead to low recall), the system uses a bootstrapping algorithm for statistical expansion. The reliable pairs identified in the first stage serve as "seeds." The algorithm then incrementally expands this set of seeds by adding new candidate hyponyms based on scoring functions. Two main scoring methodologies are explored:
  3. Graph-Based Scoring: This approach models word relationships as a graph, where words are vertices and relationships are weighted edges. It considers both direct connections between seeds and their neighbors.
  4. Simple Scoring: This method focuses on the semantic similarity between candidate hyponyms and the existing seeds.

Several edge weighting schemes are used within these scoring functions, including Inverse Document Frequency (IDF) with co-occurrence, binary weighting (presence/absence of co-occurrence), Dice coefficient, and Cosine similarity based on word vectors.

Datasets

The primary dataset used in this research is a large Turkish corpus of 490 million tokens. For evaluation, the authors manually extracted all possible hyponyms for 17 selected hypernym categories (e.g., country, city, mineral, sport) to serve as a ground truth.

Key Insights and Ideas

  • Effectiveness of Turkish Lexico-Syntactic Patterns: The study demonstrates that specific lexico-syntactic patterns can be highly effective in Turkish for initially extracting hypernym-hyponym pairs with good precision.
  • Bootstrapping for Recall Enhancement: The bootstrapping approach is crucial for expanding the initial set of pairs and significantly improving recall, moving beyond the limitations of fixed patterns.
  • Trade-off between Precision and Recall: The initial pattern-based extraction yields high precision but limited recall. The expansion phase successfully addresses the recall issue, though it requires careful selection of scoring functions to maintain precision.
  • Simplicity Can Be Effective: Surprisingly, simpler scoring methods like binary weighting (sim-bin) and co-occurrence-based scoring (sim-co) often perform as well as or better than more complex graph-based or cosine similarity methods, especially when considering computational cost.
  • Word Sense Ambiguity as a Challenge: The paper acknowledges that word sense disambiguation remains a challenge, as the expansion algorithm can be influenced by the dominant sense distribution in the corpus, potentially leading to errors for less frequent senses.

Key Results

  • The initial pattern-based extraction successfully identified over 200,000 cases, leading to the compilation of 500 reliable hypernyms.
  • The expansion algorithms, when aiming for the size of the actual hyponym list (second experiment), achieved better recall values, with some methods reaching up to 72.5%.
  • A third experiment showed that increasing the number of initial seeds generally led to better accuracy.
  • Among the tested scoring functions, gr-bin (graph-based with binary weighting), sim-bin (simple scoring with binary weighting), and sim-co (simple scoring with co-occurrence) were identified as the best-performing and most efficient.
  • The overall system successfully disclosed "is-a" relations with promising precision and recall by mining a large corpus.

Limitations and Future Work

  • Word Sense Disambiguation: The authors acknowledge that polysemy (words with multiple meanings) is a significant challenge that can affect the accuracy of the expansion algorithm.
  • Computational Cost: Graph-based and cosine similarity weighting methods were found to be costly and time-consuming.
  • Future Work: The authors plan to design a preprocessing phase to address issues arising from polysemy and other factors. They also suggest that for practical applications, a simple scoring method like sim-bin offers a good balance of performance and efficiency.

Summary updated

BibTeX
@inproceedings{yildirim2012automatic,
  author = {Savaş Yildirim and Tuğba Yildiz},
  title = {{Automatic Extraction of Turkish Hypernym-Hyponym Pairs From Large Corpus}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure