Computational Description of the Kazakh Language

Askar Dzhubanov, Bakhitdzhan KhasnovCOLING 1973

TL;DR

This paper details early computational linguistics efforts applied to the Kazakh language, specifically focusing on a statistical analysis of a major Kazakh epic. The work pioneers methods for processing and analyzing a Turkic language using computational tools, laying groundwork for broader linguistic studies in a multilingual context.

What Problem Does This Paper Address?

The paper addresses the challenge of computationally analyzing and describing the Kazakh language, a Turkic language, within the context of Kazakhstan's diverse linguistic landscape. While computational linguistics was developing, applying these methods to non-Latin script languages and languages with different structural properties presented unique hurdles. The authors aimed to develop and test methods for statistical analysis of Kazakh, particularly for understanding its structure and usage in a literary work, contributing to the broader goal of comprehensive linguistic study in a multinational society.

Methods

The core methodology involved developing automatic methods for constructing frequency lists of linguistic units using a computer. A significant challenge was adapting existing computational tools, which were largely designed for Latin and Russian alphabets, to the Kazakh language. To overcome this, the researchers adopted unit transcriptions of identical phonemes across different Turkish languages to ensure universality for machine programs. The statistical analysis employed theoretical laws such as normal, logarithmic-normal, and Poisson distributions to describe word classes.

Datasets

The primary dataset used was the epic "Abai zholy" (Abai's Path) by M. O. Auezov. This epic comprises 465,966 words (or expressions) spread across four books. The paper also mentions that the Institute is planning to statistically analyze the complete works of M. Auezov (12 volumes) and works of other leading Kazakh writers in the future.

Key Insights and Ideas

  • Adapting Computational Tools: A key insight is the necessity of adapting existing computational linguistic tools, designed for Latin/Russian scripts, to process and analyze languages like Kazakh, which have different phonetic and orthographic systems.
  • Statistical Description of a Literary Work: The paper demonstrates the feasibility and value of a comprehensive statistical description of a large literary text, providing quantitative insights into language usage.
  • Frequency of Word Classes: The analysis revealed interesting patterns in the frequency of different word classes, with conjunctions and particles appearing more frequently, suggesting their role in creating individual expressions.
  • Foundation for Multilingual Studies: The methods developed for Kazakh have practical implications for the statistical-linguistic study of other languages spoken in Kazakhstan, promoting a more inclusive approach to linguistic research.

Key Results

The statistical description of the Kazakh epic "Abai zholy" was completed, encompassing 465,966 words. The study identified and commented on 100 highly frequent wordforms, noting the prevalence of conjunctions and particles. The frequency of equally used wordforms was found to be 110. Adjectives were found to be used 38,531 times, constituting 8.3% of the total wordbuilding, with various wordforms appearing 4,773 times (7.7% of all wordforms).

Limitations and Future Work

The paper acknowledges that the statistical methods used by Kazakh linguists were, at times, applied "accidentally." Future work includes the qualitative analysis of all parts of speech in the Kazakh language, drawing from works by leading Kazakh poets and writers. The Institute is also planning to statistically analyze the complete works of M. Auezov and other prominent Kazakh writers. The developed experiment and methods are seen as having practical meaning for the statistical-linguistic study of other languages of the nations of Kazakhstan.

Summary updated

BibTeX
@inproceedings{dzhubanov1973computational,
  author = {Askar Dzhubanov and Bakhitdzhan Khasnov},
  title = {{Computational Description of the Kazakh Language}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1973},
}
Figure