Abstract
Meladel Mistica, Jey Han Lau, Timothy Baldwin. Proceedings of the Sixth International Joint Conference on Natural Language Processing.
2013. This paper demonstrates how to automatically group verbs into meaningful syntactic and semantic classes for Indonesian—a language with millions of speakers but few computational tools. By using a non-parametric Bayesian model (Hierarchical Dirichlet Processes), the authors show that we can induce "Levin-style" word classes from raw text without needing expensive pre-processing tools like syntactic parsers.
Introduction: The Challenge of Under-resourced Languages
Building a "precision grammar"—a set of rules that allow a computer to understand sentence structure deeply—requires a massive, hand-coded lexicon. For a language like Indonesian, this is a significant bottleneck. Indonesian is considered "under-resourced" in the NLP world; while it has POS taggers, it lacks the reliable syntactic parsers or "chunkers" typically used to automate lexicon building in languages like English or German.
The authors specifically target the Indonesian suffix -kan. This suffix is a "valency changer"—it can turn a simple verb into a causative or benefactive one (e.g., "buy" becomes "buy for someone"). However, how -kan behaves depends entirely on the verb stem it attaches to. Manually mapping these behaviors for every verb is exhausting, leading the authors to investigate whether unsupervised machine learning can group verbs that behave similarly.
Methodology
The researchers compare two main approaches to clustering word stems based on their "distributional similarity"—the idea that words appearing in similar contexts have similar meanings:
- Baseline (NoHDP): Standard Hierarchical Agglomerative Clustering (HAC) performed directly on raw word frequency counts in the surrounding context.
- Proposed Method (HDP): A Hierarchical Dirichlet Process (HDP) is used to model the text. HDP is a non-parametric Bayesian approach related to Latent Dirichlet Allocation (LDA). Unlike LDA, HDP discovers latent semantic topics within the corpus without requiring a pre-defined number of topics. Clustering is then performed on these topic probabilities.
Since a parser was unavailable, the researchers used "proxies" for syntax, examining unigrams in a window of 1–5 words around the target verb. Crucially, function words (prepositions, etc.) were included to help the model infer structural information.
Datasets and Evaluation
The model was trained on an Indonesian Wikipedia dump containing approximately 26 million words. Performance was evaluated against two gold standards:
- Levin Classes: 30 stems mapped to 16 classes based on VerbNet.
- Superordinate Types: 25 verb stems manually grouped into 8 broader types based on their interaction with the -kan suffix.
Results and Discussion
The HDP-based system consistently outperformed the baseline and random assignments across both evaluation sets.
Key Insights:
- Levin’s Hypothesis: The results support the "syntax-semantics interface" hypothesis—that a word's meaning is closely tied to its syntactic behavior. By clustering based on semantic "topics," the model successfully captured syntactic patterns.
- HDP as a Denoiser: Using HDP to extract latent topics before clustering significantly outperformed clustering on raw word counts. The HDP model achieved an F-score ($pF_1$) of 0.367 on verbs, more than triple the performance of the non-HDP baseline (0.111).
- Granularity Paradox: Surprisingly, the model was better at identifying fine-grained, semantically tight "Levin classes" than the broader, more abstract "superordinate" syntactic types. While the HDP model improved over the baseline for superordinate types (0.261 vs 0.152), it struggled to significantly beat a simple "majority class" baseline. This suggests that distributional models are highly sensitive to semantic coherence.
Conclusion and Future Work
This study demonstrates the viability of unsupervised induction for building lexicons in under-resourced languages. While the HDP model effectively captures semantic-syntactic relationships, the lack of explicit syntax remains a limitation. Future work may involve scaling the evaluation to larger verb sets and developing more sophisticated features to capture structural dependencies without a full parser.
Summary updated