DadmaTools: Natural Language Processing Toolkit for Persian Language

Romina Etezadi, Mohammad Karrabi, Najmeh Zare, Mohamad Bagher Sajadi, Mohammad Taher PilehvarNAACL 2022

TL;DR

DadmaTools is a new, open-source Python toolkit designed to simplify and advance Natural Language Processing (NLP) for the Persian language. It offers a comprehensive neural pipeline, integrating state-of-the-art models and providing easy access to essential Persian datasets and pre-trained embeddings, aiming to bridge the gap in readily available, high-performance tools for this under-resourced language.

What Problem Does This Paper Address?

Developing robust NLP tools for low-resource languages like Persian presents significant challenges. Existing Persian NLP toolkits often rely on older, non-neural models, limiting their ability to leverage recent advancements in deep learning. Furthermore, many are not open-source, lack comprehensive task coverage, or fail to achieve state-of-the-art performance across various fundamental NLP tasks such as tokenization, lemmatization, and parsing. This fragmentation and performance gap hinder research and practical application of NLP for Persian.

Methods

DadmaTools is built upon the spaCy framework, allowing for seamless integration with other spaCy-based pipelines. Its core is a neural pipeline that leverages fine-tuned ParsBERT models, a powerful pre-trained language model for Persian, on the PerDT dataset. Key components include:

  • Dataset Module: Facilitates automatic downloading and utilization of popular Persian NLP datasets, supporting tasks like text summarization, NER, and dependency parsing.
  • Embedding Module: Provides access to various pre-trained word and sentence embeddings for Persian, addressing a common lack of resources in this area.
  • NLP Pipeline: Offers a range of processing capabilities, including:
  • Normalization: Handles tasks like whitespace correction, character unification, and punctuation removal.
  • Tokenization, Sentence Splitting, and Multi-Word Token (MWT) Expansion: Utilizes a seq2seq architecture similar to Stanza.
  • Lemmatization: Employs a seq2seq model, with manual validation to improve performance.
  • Part of Speech (POS) Tagging & Dependency Parsing: Leverages the ACE model, fine-tuned on ParsBERT for sequence labeling and dependency parsing tasks.
  • Constituency Parsing: Uses the Supar library with a CRFConstituencyParser.
  • Chunking: Implemented with a rule-based system.
  • Ezafe Detection: A specific module trained on the Bijankhan corpus to identify this crucial Persian grammatical particle.

Datasets

DadmaTools integrates and supports a variety of Persian datasets, including:

  • PerDT: A key dataset for training and evaluation, noted for its manual curation and low error rate.
  • ARMAN: Used for tasks like NER.
  • PEYMA: Another dataset for NER.
  • FarsTail: For textual entailment.
  • FaSpell: For spell checking.
  • PersianNews: For text classification.
  • PnSummary: For text summarization.
  • SnappfoodSentiment: For sentiment classification.
  • TEP: For text translation.
  • WikipediaCorpus: A general corpus.
  • PersianTweets: A corpus of Persian tweets.
  • Seraji: A dataset with 6K instances, used in experiments.
  • Bijankhan: A large dataset (80K instances) valuable for ezafe detection.
  • Dorsa Treebank: A Persian constituency treebank used for training the constituency parser.

The paper also lists several pre-trained word embeddings, including GloVe, FastText, and word2vec, trained on various Persian corpora.

Key Insights and Ideas

  • Bridging the Gap for Persian NLP: The most significant insight is the creation of a unified, open-source toolkit that addresses the long-standing need for high-quality, modern NLP tools for Persian.
  • Leveraging ParsBERT for State-of-the-Art: The successful fine-tuning of ParsBERT on the PerDT dataset demonstrates its effectiveness for various downstream tasks, pushing the performance boundaries for Persian NLP.
  • Integrated Dataset and Embedding Management: The toolkit's design to easily download and manage datasets and embeddings is crucial for researchers and developers, significantly lowering the barrier to entry.
  • Ezafe Detection as a Dedicated Module: Recognizing the importance of ezafe in Persian grammar and creating a specific module for its detection highlights a nuanced understanding of the language's linguistic features.

Key Results

DadmaTools demonstrates competitive and, in several cases, state-of-the-art performance across various NLP tasks when compared to existing toolkits like Stanza and Hazm.

  • Dependency Parsing: DadmaTools outperforms Stanza by 1.5% for Labeled Attachment Score (LAS) and 2% for Unlabeled Attachment Score (UAS).
  • POS Tagging: Achieves better results in predicting universal tags compared to Stanza.
  • Lemmatization: Outperforms both Stanza and Hazm.
  • Constituency Parsing: Achieved an F-score of 82.88% on the Dorsa Treebank test set.
  • Speed: While Hazm is faster for POS tagging and lemmatization due to its rule-based nature, DadmaTools shows efficient run times for its neural models, with an average of 0.027 seconds per instance for dependency parsing on a GPU.

Limitations and Future Work

The authors acknowledge several limitations and propose future directions:

  • Limited High-Level Tasks: Currently, DadmaTools focuses on fundamental NLP tasks. Future work aims to extend support to higher-level tasks like sentiment analysis, entailment, and summarization.
  • Extensibility: While the toolkit is designed to be extensible, the authors plan to provide users with the ability to add new datasets and models more directly.
  • Rule-Based Chunker: The chunker is rule-based and lacks a gold dataset for evaluation, making it difficult to quantitatively assess its performance.
  • Dataset Unification Challenges: Combining datasets like PerDT and Seraji for dependency parsing and POS tagging proved challenging due to tag inconsistencies, preventing their use in the final DadmaTools model.

The authors express hope that DadmaTools will significantly contribute to research and development in Persian NLP.

Summary updated

BibTeX
@inproceedings{etezadi2022dadmatools,
  author = {Romina Etezadi and Mohammad Karrabi and Najmeh Zare and Mohamad Bagher Sajadi and Mohammad Taher Pilehvar},
  title = {{DadmaTools: Natural Language Processing Toolkit for Persian Language}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure