Command R7B Arabic: a small, enterprise-focused, multilingual, and culturally aware Arabic LLM

Yazeed Alnumay, Alexandre Barbet, Anna Bialas, William Darling, Shaan cohere.com Shaan cohere.com, Jennifer cohere.com Jennifer cohere.com, Kyle Duffy, Stephaniehowe cohere.com Stephaniehowe cohere.com, Olivia Lasche, Juin Lee, Anirudh cohere.com Anirudh cohere.com2025 · DOI

TL;DR

Researchers at Cohere have released Command R7B Arabic, a 7-billion parameter model specifically optimized for enterprise use cases in the Arabic language. By combining a novel "Multilingual Arbitrage" data strategy with an iterative post-training recipe, the model achieves state-of-the-art performance on Arabic benchmarks, outperforming larger models in cultural awareness, instruction following, and RAG (Retrieval-Augmented Generation).

What Problem Does This Paper Address?

Despite the rapid evolution of Large Language Models (LLMs), high-quality support for Arabic—the most spoken language in Africa—remains a significant challenge. The primary hurdles are:

  1. Data Scarcity: There is a limited amount of high-quality, digitized Arabic data compared to English.
  2. Linguistic Complexity: Arabic’s unique syntax, morphology, and cultural nuances often result in "awkward" or nonsensical outputs when models rely on literal translations from English.
  3. Enterprise Needs: Professional applications require specific capabilities like strict instruction following and "faithfulness" to provided context (RAG), which general-purpose multilingual models often lack in non-English languages.
  4. Resource Constraints: There is a constant need for smaller, more efficient models (like the 7B class) that can be deployed cost-effectively without sacrificing performance.

Methods

The authors built upon the existing Command R7B base and applied a sophisticated three-stage post-training pipeline:

  • Iterative Supervised Fine-Tuning (SFT): Instead of a single training pass, they used a "human-in-the-loop" approach. They defined a base data mix and iteratively added new datasets, evaluating the model at each step. If a dataset didn't improve performance, they used Multilingual Arbitrage—leveraging strong English models to refine Arabic prompts and completions—before trying again.
  • Expert Model Merging: The team trained specialized "expert" models for different domains (e.g., math, instruction following). These experts were then merged into a single generalist model using parameter-wise linear interpolation. This allowed them to enhance specific capabilities without the massive compute cost of retraining from scratch.
  • Two-Stage Preference Tuning: They used Direct Preference Optimization (DPO) in two phases: first, an offline stage for general conversational fluency, followed by an iterative on-policy stage focused on reasoning and math.

Datasets

The training and evaluation utilized a mix of public and internal datasets:

  • Training: Included ArMATH (math word problems), ArabicaQA (question answering), and synthetic data generated via the Multilingual Arbitrage process.
  • Arabic Benchmarks: The model was tested on AlGhafa-Native (cultural knowledge), TyDiQA-GoldP (reading comprehension), and ArabicMMLU (general knowledge).
  • New/Adapted Benchmarks: The authors introduced IFEval AR (an Arabic translation of the Instruction Following Evaluation) and FaithEval Arabic (measuring RAG performance and contextual faithfulness).
  • Human Preference: They used mArenaHard, an Arabic version of the LMSYS Arena "Hard" tasks, to measure how often humans (or high-quality LLM judges) prefer the model's output over competitors.

Key Insights and Ideas

  • Arabic is not "English with a Filter": The authors found that lexical control tasks (like "write exactly 300 words") don't translate literally. They collaborated with expert annotators to create language-specific instructions, such as "add N diacritics to the response," to ensure the model understood Arabic linguistic nuances.
  • The Power of Iteration: The "Multilingual Arbitrage" loop proved that synthetic data isn't just a fallback for missing data; when filtered by a reward model and human-in-the-loop oversight, it can actually drive a model to state-of-the-art performance.
  • Maintaining Generalism: A common fear in fine-tuning is "catastrophic forgetting"—losing English skills while learning Arabic. By using model merging and careful data balancing, Command R7B Arabic retained its core English capabilities while gaining Arabic mastery.

Key Results

Command R7B Arabic consistently outperformed similarly sized peers (including Llama 3.1 8B, Gemma 9B, and Qwen 2.5 7B) across the board:

  • Arabic Benchmarks: It achieved an average score of 69.3, surpassing the base Command R7B (65.8) and Gemma 9B (67.0).
  • Instruction Following: On IFEval AR, it scored 69.0, a massive jump from the base model's 57.8, proving the effectiveness of their SFT recipe.
  • Win Rates: In head-to-head comparisons on mArenaHard, Command R7B Arabic achieved an 84.8% win rate against Llama 3.1 8B and an 84.2% win rate against Jais 30B (a much larger model).
  • RAG Performance: It led its class in FaithEval Arabic (51.6), making it particularly suitable for enterprise "chat-with-your-data" applications.

Limitations and Future Work

  • Dialectal Diversity: The current work focuses primarily on Modern Standard Arabic (MSA). While MSA is the standard for professional settings, real-world use cases often involve regional dialects (Egyptian, Levantine, Gulf, etc.), which the authors identify as a key area for future adaptation.
  • Knowledge Transfer: While the model is highly capable, the authors note that transferring deep factual knowledge from English-centric pre-training to Arabic remains an open research challenge.
  • Validation: The authors acknowledge that while benchmarks are strong proxies, the model's effectiveness in complex, multi-step enterprise workflows still requires real-world user feedback for full validation.

Summary updated

Abstract

Yazeed Alnumay, Alexandre Barbet, Anna Bialas, William Michael Darling, [email protected] [email protected], [email protected] [email protected], Kyle Duffy, [email protected] [email protected], Olivia Lasche, Justin Seonyong Lee, [email protected] [email protected], [email protected] [email protected]. Proceedings of the Sixth Workshop on African Natural Language Processing (AfricaNLP 2025). 2025.
BibTeX
@misc{alnumay2025command,
  author = {Yazeed Alnumay and Alexandre Barbet and Anna Bialas and William Darling and Shaan cohere.com Shaan cohere.com and Jennifer cohere.com Jennifer cohere.com and Kyle Duffy and Stephaniehowe cohere.com Stephaniehowe cohere.com and Olivia Lasche and Juin Lee and Anirudh cohere.com Anirudh cohere.com and Jennifer cohere.com Jennifer cohere.com},
  title = {{Command R7B Arabic: a small, enterprise-focused, multilingual, and culturally aware Arabic LLM}},
  year = {2025},
  doi = {10.18653/v1/2025.africanlp-1.17},
}
Figure