Saidzoda-Tajik-LFM-2.6B: Adapting a 2.6B Hybrid Model to Tajik Tool Calling, and What Continued Pretraining Takes Away

Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon ShodyevaarXiv 2026 · arXiv

This report details the development of Saidzoda-Tajik-LFM-2.6B, a specialized adaptation of Liquid AI’s hybrid model designed to bring tool-calling capabilities to the Tajik language. The authors demonstrate that while continued pretraining on raw text is necessary for low-resource language adaptation, it comes with a significant "tax" on specialized skills that must be carefully recovered through fine-tuning.

TL;DR

The authors adapted a 2.6B parameter hybrid model to Tajik tool calling, raising accuracy from 20.69% to 64.14% while maintaining English performance. A key finding is that continued pretraining (CPT) on raw text temporarily "breaks" specialized skills like tool calling, which then require targeted supervised fine-tuning (SFT) with multi-lingual replay to restore.

What Problem Does This Paper Address?

Tajik, a Cyrillic-script language with unique characters (ғ, ӣ, қ, ӯ, ҳ, ҷ), is rarely supported by open-source LLMs. While large models exist for general Tajik chat, there is a gap for small, efficient models that can run on modest hardware to perform tool calling—the ability to turn a natural language request into a structured API call.

Existing models like LFM2.5-2.6B perform well in English tool calling (~76%) but fail significantly when prompted in Tajik (~21%). The challenge lies in teaching the model a new language without losing the complex reasoning required for function calling.

Methods

The researchers employed a two-stage adaptation recipe using LoRA (Low-Rank Adaptation) with rank-stabilized scaling:

  1. Continued Pretraining (CPT): The model was trained for 3 epochs on ~2.43 billion tokens of Tajik web text. This stage was intended to teach the model the Tajik language.
  2. Supervised Fine-Tuning (SFT): The model was then trained on a specific "mix" designed to balance new language acquisition with skill retention:
  3. 40% Tajik Tool Calls: English datasets (Glaive, Hermes, ToolACE) machine-translated into Tajik.
  4. 35% English Replay: The original English tool-calling data to prevent catastrophic forgetting.
  5. 25% Tajik Instructions: General instruction data to maintain conversational ability in Tajik.

To evaluate the model, they used both strict parsing (matching the exact native format) and lenient parsing (accepting JSON or XML formats) to distinguish between the model forgetting a format versus forgetting the skill itself.

Datasets

  • Pretraining: A massive crawl of Tajik web documents (1.48M documents) from sources like HPLT 2.0, FineWeb-2, and Tajik Wikipedia.
  • Fine-tuning: 176,532 rows of tool-calling and instruction data.
  • Evaluation: The authors introduced a new Tajik tool-calling test set consisting of 174 items adapted from the Berkeley Function Calling Leaderboard (BFCL) v3, with user turns machine-translated into Tajik while keeping API schemas in English.

Key Insights and Ideas

  • The CPT "Skill Tax": Raw-text CPT is a double-edged sword. While it improved Tajik knowledge, it initially drove tool-calling accuracy to zero. Lenient parsing revealed that CPT didn't just break the output format; it actively degraded the underlying logic of selecting the right tool.
  • Recovery via Replay: The SFT stage successfully restored the tool-calling skill. Crucially, including English "replay" data ensured that the model's original English capabilities remained intact (76.55% base vs 77.24% final).
  • Tokenizer Readiness: A vital first step was confirming the tokenizer could represent Tajik's unique letters. Fortunately, the LFM2.5 tokenizer already had single-token representations for these characters, though Tajik text remains roughly twice as "expensive" in tokens per word compared to English.
  • Format vs. Skill: By using a lenient parser, the authors could see that the base model actually "knew" more Tajik than the strict scores suggested—it just often answered in non-standard formats (like XML or JSON) that a strict production parser would reject.

Key Results

The primary success was the massive jump in Tajik tool-calling performance.

  • Tajik Tool Calling: Strict accuracy rose from 20.69% to 64.14%.
  • English Tool Calling: Remained stable, moving from 76.55% to 77.24%.
  • General Knowledge: Tajik knowledge benchmark scores improved from 28.0 to 35.0. However, this came at the cost of English general knowledge, with English MMLU dropping from 60.8 to 52.5.

The progression of accuracy across training stages is clearly visible in the authors' evaluation:

Figure 2: Strict full-call exact accuracy by stage, En- glish and Tajik, with 95% bootstrap intervals.

The breakdown by category shows that while single and parallel calls improved significantly, multi-turn conversations remain a challenge for a model of this size:

Table 2: Per-category full-call exact accuracy (%) for the stock base and the released model (SFT epoch 1), with 95% bootstrap intervals; overall and irrelevance rows below. Intervals for small categories are wide. Greedy decoding.

Limitations and Future Work

  • Abstention Weakness: The model struggles to say "no." If no tool fits the request, the model often tries to force a call anyway (irrelevance accuracy dropped to 6.9% in Tajik). The authors attribute this to a lack of "no-call" examples in the SFT data.
  • Multi-turn Logic: Performance on multi-turn tool calling remains low (~17% in Tajik), suggesting that 2.6B parameters may be tight for complex stateful reasoning.
  • Machine Translation Bias: The test set relies on machine translation. Real-world Tajik, with its various dialects and informal spellings, may yield different results.
  • Future Directions: The team plans to explore preference or reinforcement training (RLHF/DPO) to address the abstention and multi-turn weaknesses.

Summary updated

Abstract

We release Saidzoda-Tajik-LFM-2.6B, a Tajik adaptation of the 2.6B-parameter hybrid model LFM2.5-2.6B-Base, built for tool calling in Tajik on modest hardware. The recipe has two stages: LoRA continued pretraining (CPT) on 1,485,319 cleaned Tajik web documents for 3 epochs, then one epoch of supervised fine-tuning (SFT) on a mix of machine-translated Tajik tool calls, English tool calls kept as replay, and Tajik instructions. On a 174-item tool-calling test set built from BFCL v3 and machine-translated to Tajik, strict full-call accuracy on Tajik prompts rises from 20.69 for the base model to 64.14 (paired difference +43.45 points, 95% interval +35.2 to +51.7); a lenient parser that also accepts the base model's own call formats puts the base at 31.72. English shows no detectable change (76.55 and 77.24). Every CPT checkpoint scores zero: CPT removes the native tool-call format and, as the lenient re-scoring shows, part of the skill itself. Outside tool calling, accuracy on a Tajik knowledge benchmark rises from 28.0 to 35.0, Belebele Tajik shows no significant change, and English MMLU falls from 60.8 to 52.5. The model is not a general Tajik assistant. It almost never declines when no tool fits (6.9% correct on Tajik, against 31.0% for the base under the lenient parser), it is weak at multi-turn calls, and its calls should be checked before they are executed. We describe the data, the training settings and every evaluation in enough detail to repeat them.
BibTeX
@misc{saidzoda2026saidzoda,
  author = {Tohir Saidzoda and Nekruz Maxudov and Bedil Karimov and Abubakr Abdullozoda and Amin Jalilov and Jonibek Kishvarov and Rahmonali Sharipov and Mariyamkhon Shodyeva},
  title = {{Saidzoda-Tajik-LFM-2.6B: Adapting a 2.6B Hybrid Model to Tajik Tool Calling, and What Continued Pretraining Takes Away}},
  year = {2026},
  eprint = {2610.41576},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.41576},
}
Figure