Abstract
We investigate the problem of learning a part-of-speech (POS) lexicon for a resource-poor language, dialectal Arabic. Developing a high-quality lexicon is often the first step towards building a POS tagger, which is in turn the front-end to many NLP systems. We frame the lexicon acquisition problem as a transductive learning problem, and perform comparisons on three transductive algorithms: Transductive SVMs, Spectral Graph Transducers, and a novel Transductive Clustering method. We demonstrate that lexicon learning is an important task in resource-poor domains and leads to significant improvements in tagging accuracy for dialectal Arabic.
Introduction
Part-of-Speech (POS) tagging is a foundational task in Natural Language Processing (NLP), yet it remains notoriously difficult for "resource-poor" languages. This paper focuses on Levantine Colloquial Arabic (LCA), a dialect that, unlike Modern Standard Arabic (MSA), lacks large annotated corpora or specialized morphological analyzers.
Standard unsupervised taggers, such as Hidden Markov Models (HMMs), rely heavily on a POS-set lexicon—a comprehensive list of all possible tags for every word in a vocabulary. This lexicon acts as a set of "hard constraints" that guide the tagger. When a lexicon is incomplete or overly "noisy" (suggesting too many possible tags), tagger accuracy drops significantly. In LCA, existing MSA tools fail to analyze approximately 23% of the words. This research aims to "fill in the blanks" of this partial lexicon using machine learning.
The Lexicon Acquisition Pipeline
The authors propose a three-stage process to move from raw text to a functional POS tagger.
- Initial Analysis: Raw text is processed using the Buckwalter Morphological Analyzer (designed for MSA) to generate a partial POS lexicon.
- Transductive Learning: Machine learning models infer the possible tags for "un-analyzable" words, resulting in a full POS lexicon.
- HMM Training: The full lexicon is used to train an HMM tagger via Expectation-Maximization (EM).
Methodology: Transductive Learning and Compound Labels
The core innovation lies in framing lexicon acquisition as a transductive learning problem. Unlike inductive learning, which builds a general model to predict unseen data, transductive learning uses both labeled and unlabeled data simultaneously to predict labels for a specific, known set of unlabeled points—in this case, the un-analyzable words in the corpus.
Compound-Label Framework
Instead of predicting individual tags for a word, the system treats each unique set of tags (e.g., {Noun, Verb}) as a single atomic label. This approach captures dependencies between tags; for instance, if a word can be a Noun, it might also frequently be an Adjective.
The researchers compared several algorithms:
- Transductive SVM (TSVM): Maximizes the margin on both labeled and unlabeled data.
- Spectral Graph Transducer (SGT): A graph-based approach partitioning data based on similarity.
- Transductive Clustering (TC): A novel method introduced in this paper that uses a PAC-Bayesian bound to automatically select clustering parameters.
- Inductive SVM (ISVM): Used as a baseline.
Experimental Setup
The study utilized 476k tokens of raw text from the Fisher collection of Levantine Colloquial Arabic conversational telephone speech. Evaluation was performed on 15k tokens manually annotated with POS tags using the Bies tagset. Features used for classification included orthographic features (prefixes and suffixes) and contextual features (neighboring words and tags).
Key Results and Insights
The machine-learned lexicons were compared against two baselines: one where un-analyzable words could be any "open class" tag, and another where they could be "any" tag.
- Tagging Accuracy: The Transductive SVM (TSVM) was the top performer, achieving 63.54% overall tagging accuracy, a significant improvement over the "all tags" baseline of 55.64%.
- The Power of Constraints: A lexicon's primary value in unsupervised learning is eliminating incorrect tags. Machine-learned lexicons were much more "constrained," averaging 1.8 to 1.9 tags per word, compared to 5.17 in the baseline.
- Robustness: The system proved relatively robust to errors in the initial MSA-derived lexicon. Even with imperfect "seed" data, the transductive process yielded a net gain in tagging accuracy.
Limitations and Future Work
While successful, the approach faces challenges such as error propagation from the initial analyzer. Future work could involve improving the EM optimization for the HMM tagger to better handle dialectal noise and incorporating more sophisticated linguistic features beyond simple orthographic and contextual cues.
Summary updated