Saidzoda-CA-EmbeddingGemma-300M: One Small Embedder for Tajik, Kazakh, Uzbek and Kyrgyz

Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon ShodyevaarXiv 2026 · arXiv

TL;DR

Researchers from Saidzoda AI Lab have developed a single, compact 300M-parameter text embedding model that effectively serves four Central Asian languages: Tajik, Kazakh, Uzbek, and Kyrgyz. By fine-tuning with a mix of synthetic "grounded" questions and "replay" data, they achieved massive performance gains (up to +26 points) over base models, proving that one small model can efficiently handle a linguistically diverse region.

What Problem Does This Paper Address?

Building retrieval systems for Central Asia is a logistical and computational headache. The region uses a mix of language families—Tajik (Persian/Indo-European) and Kazakh, Uzbek, and Kyrgyz (Turkic)—written in different scripts (Cyrillic and Latin).

Standard practice often involves serving one specialized model per language, which quadruples the infrastructure costs. Meanwhile, small, open-source "off-the-shelf" embedders like Google's EmbeddingGemma-300M perform poorly on these languages, often failing to find the correct passage more than half the time. The authors set out to see if a single, tiny model could serve the entire region without sacrificing the performance of language-specific models.

Methods

The team started with a version of EmbeddingGemma-300M already fine-tuned for Tajik. They then performed one additional epoch of training using a carefully curated mix of data:

  • Synthetic Data Generation: They used a 27B-parameter Qwen model to write "grounded" questions based on source passages (Wikipedia, news, etc.) in Kazakh, Uzbek, and Kyrgyz. A second model then verified these pairs to ensure accuracy.
  • Training Recipe: They employed in-batch multiple-negatives loss with gradient caching, allowing for larger effective batch sizes on a single GPU. They also mined "hard negatives"—passages that are similar to the correct one but don't actually answer the question—to sharpen the model's retrieval skills.
  • Catastrophic Forgetting Prevention: To ensure the model didn't "forget" Tajik or its original English/Russian capabilities, they included "refresher" Tajik data and "replay" sets from English and Russian benchmarks.

Datasets

The training set consisted of 279,904 pairs, broken down as follows:

The training data mix used for the four-language model.

For evaluation, the authors used:

  • Belebele: A parallel reading comprehension dataset used as a retrieval benchmark.
  • Wikipedia Test Sets: 500 generated questions per Turkic language on held-out articles.
  • Domain-Specific Tajik Sets: Specialized sets for Tajik law and tax documents.

Key Insights and Ideas

  • The "One Model" Success: The unified model performed nearly as well as specialized single-language models (within 2.8 points). This suggests that for many practical applications, the efficiency of serving one model outweighs the marginal gains of language-specific ones.
  • Cross-Lingual Transfer: Interestingly, training on Tajik alone improved performance on the Turkic languages. This is likely due to shared cultural context, loanwords, and the model learning the general "style" of retrieval tasks.
  • Small Models vs. Giants: While the 300M model still trails larger 560M models (like XLM-RoBERTa based E5) on general benchmarks, it actually outperforms them on specific local domains like Tajik law. This highlights the power of targeted fine-tuning for low-resource languages.

Key Results

The model, named Saidzoda-CA-EmbeddingGemma-300M, showed dramatic improvements over the base EmbeddingGemma-300M model.

Belebele recall@1 comparison showing the unified model (dark blue) significantly outperforming the released base model (light grey).
  • Belebele Recall@1: Gains ranged from +14.4 to +26.0 points per language, reaching a final accuracy of ~75-78%.
  • Mean Performance: The average recall across all four languages jumped from 58.0% to 76.0%.
  • Tajik Domain Retention: The model successfully maintained its high performance on Tajik legal documents (92.2% recall@10), proving the "refresher" strategy worked.

Limitations and Future Work

  • Synthetic Bias: Much of the evaluation data was generated by the same family of models used for training. The authors acknowledge these scores should be viewed as "upper bounds."
  • The Turkic Gap: The model still trails larger 560M-parameter models on Turkic language benchmarks by about 10 points. Closing this gap while staying at the 300M scale remains a challenge.
  • Script Coverage: The Uzbek evaluation was almost entirely in Latin script; the model's performance on Cyrillic Uzbek remains largely untested.
  • Future Directions: The authors suggest testing with real-world user queries and exploring why the performance gap between small and large models persists more strongly in Turkic languages than in Tajik.

Summary updated

Abstract

The small open embedder EmbeddingGemma-300M, used as released, finds the right passage first for only 51.7% to 60.8% of Belebele questions in Tajik, Kazakh, Uzbek and Kyrgyz. We present Saidzoda-CA-EmbeddingGemma-300M, one model for all four languages. It is our Tajik fine-tune of EmbeddingGemma-300M, trained for one more epoch on 279,904 pairs: questions in Kazakh, Uzbek and Kyrgyz that our own language model wrote about source passages, a Tajik refresher, and English and Russian replay. On Belebele, recall@1 rises by 14.4 to 26.0 points in each language, to between 75.2% and 77.7%. Against single-language models trained with the same settings, the one model stays within 2.8 points at recall@1 in Kazakh, Uzbek and Tajik, with overlapping recall@10 intervals, though it also had more training. It keeps the Tajik law recall@10 of its starting point (92.2 against 92.4) but loses 4.4 points of law recall@1. A model trained on one of these languages also improves the others on Belebele, but less than training on them. The model still trails the 560M XLM-RoBERTa embedders on Kazakh, Uzbek and Kyrgyz Belebele: the better of multilingual-e5-large-instruct and BGE-M3, used as released, is 9.4 to 10.2 points ahead at recall@1. Our scores of those released models agree with the published MTEB results within 2.4 points. Russian and English recall@1 fall by 3.0 and 4.2 points.
BibTeX
@misc{saidzoda2026saidzoda,
  author = {Tohir Saidzoda and Nekruz Maxudov and Bedil Karimov and Abubakr Abdullozoda and Amin Jalilov and Jonibek Kishvarov and Rahmonali Sharipov and Mariyamkhon Shodyeva},
  title = {{Saidzoda-CA-EmbeddingGemma-300M: One Small Embedder for Tajik, Kazakh, Uzbek and Kyrgyz}},
  year = {2026},
  eprint = {2611.11471},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11471},
}
Figure