TL;DR
This paper introduces a novel data-centric approach to improve reasoning in Large Language Models (LLMs) for specialized, non-English domains like medicine and cybersecurity. By synthesizing "code-switched" tasks—where English and a target language are mixed within a single problem—the authors create a "cross-lingual bridge" that allows models to apply their strong English-side reasoning capabilities to other languages, even when native training data is scarce.
What Problem Does This Paper Address?
Reinforcement Learning with Verifiable Rewards (RLVR) has recently supercharged the reasoning abilities of LLMs (think DeepSeek-R1). However, RLVR relies on verifiable rewards—tasks where an answer can be objectively checked, like math problems or code execution.
This creates two major bottlenecks:
- Domain Gap: In knowledge-intensive fields like medicine or cybersecurity, most data is "unverifiable" raw text (textbooks, blogs, papers) rather than neat Q&A pairs.
- Language Gap: While English domain text is abundant, high-quality, verifiable reasoning data in other languages (like Korean, Swahili, or Hindi) is incredibly rare.
Existing methods often rely on simple translation, but this frequently fails because models struggle to map non-English inputs to their internal "reasoning pathways," which are typically dominant in English.
Methods
The authors propose a pipeline that transforms raw English text into a cross-lingual training signal for RLVR.
- Verifiable FIM-MCQA Synthesis: They take raw English domain text and use an LLM to extract an educational passage. They then mask a specific span of text and treat it as the "correct answer," generating plausible distractors to create a Fill-In-the-Middle Multiple-Choice Question (FIM-MCQA). Because the answer is a verbatim substring of the original text, it is automatically verifiable.
- Code-Switched Task Synthesis: Instead of just translating these tasks, they create "code-switched" variants. This includes:
- Option-side Code-Switching: An English passage paired with target-language answer options.
- Passage-level Code-Switching: Mixing English and target-language spans within the passage itself.
- RLVR Training: They use these synthesized tasks to fine-tune models (Llama 3.2, Qwen 2.5, and SmolLM3) using Direct Alignment from Preference Optimization (DAPO).
Datasets
- Training Source:
- Cybersecurity: The Primus dataset (500k documents from MITRE, Wikipedia, etc.).
- Medical: The Biomed-Enriched dataset (450k documents from PubMed).
- Evaluation Benchmarks:
- Medical: MedQA, MedMCQA, and native Korean benchmarks like KorMedMCQA-Doctor and KMMLU-Pro-Medical.
- Cybersecurity: SecEval, CyberMetric, and CTI-MCQ, evaluated across Korean, Swahili, Hindi, and Japanese.
Key Insights and Ideas
- The "Bridge" Effect: Code-switching acts as a training scaffold. By forcing the model to process English and a target language in the same context to reach a single rewardable answer, the model learns to "bridge" the two languages internally.
- Input Understanding is the Bottleneck: The authors found that a model's failure in non-English reasoning often starts at the input stage. Code-switched training significantly improves the model's ability to recover English-side semantic meaning from target-language inputs (see Figure 2).
- Middle-Layer Alignment: Mechanistic analysis showed that code-switched training doesn't just change the output; it actually aligns the model's internal representations in the middle layers (layers 12–24). This suggests the model is learning to map different languages into a shared, language-agnostic "reasoning space."
Key Results
- Superior Target Performance: On Korean medical benchmarks, the code-switched approach (OURS) consistently outperformed English-only, Target-only, and Bilingual-mixture baselines. For example, using Qwen2.5-3B, the average target-language score improved from 35.01 (Base) to 35.94.
- Cross-Lingual Consistency: The method significantly reduced "answer asymmetry"—cases where a model gets the English version of a question right but the Korean version wrong.
- Zero-Shot Transfer: The benefits extended to low-resource languages like Swahili and Hindi in the cybersecurity domain, proving the method's robustness across different scripts and resource levels.
- No "Tax" on English: Crucially, these gains in target languages came without degrading the model's original English performance.
Limitations and Future Work
- Compute Intensity: RLVR is expensive. Each training run required roughly 2,500 H100 GPU hours, limiting the authors' ability to test larger models or more exhaustive hyperparameter searches.
- Evaluation Coverage: While the results are promising, the authors acknowledge that many target-language benchmarks are themselves translated from English, which might introduce artifacts.
- Language Scope: The study focused heavily on Korean. Future work is needed to see if these "bridges" hold up as effectively for even lower-resource languages or more diverse language families.
Summary updated