TL;DR
This paper tackles the challenge of creating a comprehensive Chinese thesaurus that accounts for regional variations. By leveraging an existing thesaurus (Tongyici Cilin) and using automatic classification methods on regional corpora, they demonstrate a promising approach to building a Pan-Chinese lexical resource, achieving over 85% accuracy in some cases.
What Problem Does This Paper Address?
Building a robust lexical resource for a language as widely spoken and regionally diverse as Chinese is a significant challenge. Existing Chinese thesauri, like the widely used Tongyici Cilin, are often based on language use in a single region (primarily Mainland China) and thus lack comprehensiveness. This leads to a gap in capturing the substantial regional variations in vocabulary and meaning, which is crucial for various Natural Language Processing (NLP) tasks like understanding, information retrieval, and machine translation. The paper aims to bridge this gap by extending an existing thesaurus with region-specific words from different Chinese speech communities.
Methods
The core of the paper's methodology involves using an existing semantic classificatory structure (Tongyici Cilin) as a foundation and automatically classifying new, region-specific words into this structure.
- Feature Extraction: They extract feature vectors for both target words and Cilin subclasses. For target words, these vectors are derived from their co-occurring content words within a ±5 word window in a corpus. For Cilin subclasses, feature vectors are created by taking the union of features from all individual members within that subclass.
- Similarity Measurement: The cosine similarity function is employed to measure the similarity between the feature vectors of target words and Cilin subclasses. They experiment with two versions: binary vectors and real-valued vectors.
- Classification: Target words are classified into Cilin subclasses based on their cosine similarity scores.
- Data Source for Cilin Features: A key aspect of their experimentation is investigating whether using feature vectors for Cilin subclasses extracted from the same region as the target word (e.g., Hong Kong data for Hong Kong words) or from a general Mainland China subcorpus (Beijing) yields better results.
- Baseline: A simple baseline method is used for comparison, ranking subclasses by their size (number of words).
Datasets
- Tongyici Cilin (同義詞詞林): The existing Chinese thesaurus used as the target classification structure. It contains approximately 70,000 Chinese lexical items organized hierarchically.
- LIVAC Synchronous Corpus: A large corpus collected from six Chinese speech communities (Hong Kong, Beijing, Taipei, Singapore, Shanghai, Macau). For this study, they specifically used the financial news sections from the Beijing (BJ), Hong Kong (HK), Singapore (SG), and Taipei (TW) subcorpora, covering the period 1995-2004.
- Test Data: Unique lexical items (approximately 30-40% of which are not covered by Cilin) were extracted from the LIVAC subcorpora. The study sampled the 30 most frequent words from each of the Beijing, Hong Kong, Singapore, and Taipei lists as target words for classification. These words were not present in the Tongyici Cilin.
Key Insights and Ideas
- Regional Specificity is a Major Hurdle: The study highlights that regional variations in language use pose a significant challenge for creating a unified Pan-Chinese lexicon. Human judges also struggled with unfamiliar regional terms, underscoring the difficulty of manual construction.
- Automatic Classification is Promising: Despite the challenges, automatic classification methods, particularly using cosine similarity with appropriate feature extraction, show considerable promise for extending existing thesauri.
- Feature Vector Source Matters: The choice of data source for extracting feature vectors for Cilin subclasses can significantly impact classification performance, especially for region-specific target words. Using features from the same region as the target word often yielded better results than using a general Mainland China corpus.
- Real-valued Vectors Outperform Binary: Using real-valued vectors for similarity measurement generally led to better results, likely because they are less affected by differences in feature space size and occurrence counts.
- Human Judgement is Difficult but Valuable: While creating a gold standard through human annotation is challenging due to regional biases and unfamiliarity, the inter-annotator agreement was satisfactory (Kappa values around 0.6-0.7), validating the human judgments as a reliable benchmark.
Key Results
- High Accuracy Achieved: Automatic classification achieved accuracies of over 85% in some cases when considering the top 15 candidates, significantly outperforming the baseline.
- Hong Kong Data Shows Strong Performance: For Hong Kong target words, accuracy reached 87% with the top 15 candidates and over 95% with the top 20, even when using features from the Beijing subcorpus.
- Taipei Data Most Affected by Heterogeneity: Taipei data showed the most significant impact from different testing conditions, suggesting that the Cilin might not have adequate contextual feature vectors extracted from the Taipei subcorpus.
- Inter-Annotator Agreement: Kappa statistics for human judgment were K=0.6870 at the subclass level and K=0.5971 at the semantic head level, both statistically significant.
Limitations and Future Work
- Limited Test Data Size: The study used a relatively small sample of 30 target words per region. Future work will involve expanding the test data size.
- Focus on Financial Domain: The current study focused on the financial domain. Future work will explore classifying words from other domains like sports and the general domain.
- Classification at Semantic Head Level: The current study focused on subclass level classification. Future work will experiment with the more fine-grained semantic head level of the Cilin structure.
- Part-of-Speech: The study did not explicitly analyze the impact of part-of-speech on classification. Future work will investigate this aspect.
- Alternative Methods: Exploring alternative methods like nearest neighbor approaches is suggested.
- Dialectal Variation: The study primarily dealt with Modern Standard Chinese. Expansion to handle various dialects is a potential future direction.
Summary updated