Lexicon Acquisition for Dialectal Arabic Using Transductive Learning

Kevin Duh, Katrin KirchhoffEMNLP 2006 · OpenReview

Abstract

We investigate the problem of learning a part-of-speech (POS) lexicon for a resource-poor language, dialectal Arabic. Developing a high-quality lexicon is often the first step towards building a POS tagger, which is in turn the front-end to many NLP systems. We frame the lexicon acquisition problem as a transductive learning problem, and perform comparisons on three transductive algorithms: Transductive SVMs, Spectral Graph Transducers, and a novel Transductive Clustering method. We demonstrate that lexicon learning is an important task in resource-poor domains and leads to significant improvements in tagging accuracy for dialectal Arabic.

Introduction

Part-of-Speech (POS) tagging is a foundational task in Natural Language Processing (NLP), yet it remains notoriously difficult for "resource-poor" languages. This paper focuses on Levantine Colloquial Arabic (LCA), a dialect that, unlike Modern Standard Arabic (MSA), lacks large annotated corpora or specialized morphological analyzers.

Standard unsupervised taggers, such as Hidden Markov Models (HMMs), rely heavily on a POS-set lexicon—a comprehensive list of all possible tags for every word in a vocabulary. This lexicon acts as a set of "hard constraints" that guide the tagger. When a lexicon is incomplete or overly "noisy" (suggesting too many possible tags), tagger accuracy drops significantly. In LCA, existing MSA tools fail to analyze approximately 23% of the words. This research aims to "fill in the blanks" of this partial lexicon using machine learning.

The Lexicon Acquisition Pipeline

The authors propose a three-stage process to move from raw text to a functional POS tagger.

The three-stage pipeline for lexicon acquisition and POS tagging.
  1. Initial Analysis: Raw text is processed using the Buckwalter Morphological Analyzer (designed for MSA) to generate a partial POS lexicon.
  2. Transductive Learning: Machine learning models infer the possible tags for "un-analyzable" words, resulting in a full POS lexicon.
  3. HMM Training: The full lexicon is used to train an HMM tagger via Expectation-Maximization (EM).

Methodology: Transductive Learning and Compound Labels

The core innovation lies in framing lexicon acquisition as a transductive learning problem. Unlike inductive learning, which builds a general model to predict unseen data, transductive learning uses both labeled and unlabeled data simultaneously to predict labels for a specific, known set of unlabeled points—in this case, the un-analyzable words in the corpus.

Compound-Label Framework

Instead of predicting individual tags for a word, the system treats each unique set of tags (e.g., {Noun, Verb}) as a single atomic label. This approach captures dependencies between tags; for instance, if a word can be a Noun, it might also frequently be an Adjective.

The compound-label framework treats sets of potential tags as single atomic labels for classification.

The researchers compared several algorithms:

  • Transductive SVM (TSVM): Maximizes the margin on both labeled and unlabeled data.
  • Spectral Graph Transducer (SGT): A graph-based approach partitioning data based on similarity.
  • Transductive Clustering (TC): A novel method introduced in this paper that uses a PAC-Bayesian bound to automatically select clustering parameters.
  • Inductive SVM (ISVM): Used as a baseline.

Experimental Setup

The study utilized 476k tokens of raw text from the Fisher collection of Levantine Colloquial Arabic conversational telephone speech. Evaluation was performed on 15k tokens manually annotated with POS tags using the Bies tagset. Features used for classification included orthographic features (prefixes and suffixes) and contextual features (neighboring words and tags).

Key Results and Insights

The machine-learned lexicons were compared against two baselines: one where un-analyzable words could be any "open class" tag, and another where they could be "any" tag.

  • Tagging Accuracy: The Transductive SVM (TSVM) was the top performer, achieving 63.54% overall tagging accuracy, a significant improvement over the "all tags" baseline of 55.64%.
  • The Power of Constraints: A lexicon's primary value in unsupervised learning is eliminating incorrect tags. Machine-learned lexicons were much more "constrained," averaging 1.8 to 1.9 tags per word, compared to 5.17 in the baseline.
  • Robustness: The system proved relatively robust to errors in the initial MSA-derived lexicon. Even with imperfect "seed" data, the transductive process yielded a net gain in tagging accuracy.

Limitations and Future Work

While successful, the approach faces challenges such as error propagation from the initial analyzer. Future work could involve improving the EM optimization for the HMM tagger to better handle dialectal noise and incorporating more sophisticated linguistic features beyond simple orthographic and contextual cues.

Summary updated

Abstract

We investigate the problem of learning a part-of-speech (POS) lexicon for a resource-poor language, dialectal Arabic. Developing a high-quality lexicon is often the first step towards building a POS tagger, which is in turn the front-end to many NLP systems. We frame the lexicon acquisition problem as a transductive learning problem, and perform comparisons on three transductive algorithms: Transductive SVMs, Spectral Graph Transducers, and a novel Transductive Clustering method. We demonstrate that lexicon learning is an important task in resource-poor domains and leads to significant improvements in tagging accuracy for dialectal Arabic.
BibTeX
@inproceedings{duh2006lexicon,
  author = {Kevin Duh and Katrin Kirchhoff},
  title = {{Lexicon Acquisition for Dialectal Arabic Using Transductive Learning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  url = {https://openreview.net/forum?id=rJZXsGM_-B},
}
Figure