From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis

Cheonbok Park, Jihwan Kim, Shinyoung Joo, Joosung Lee, Hwiyeol Jo, Jeonghoon Kim, Jaegul ChooarXiv 2026 · arXiv

TL;DR

This paper introduces a novel data-centric approach to improve reasoning in Large Language Models (LLMs) for specialized, non-English domains like medicine and cybersecurity. By synthesizing "code-switched" tasks—where English and a target language are mixed within a single problem—the authors create a "cross-lingual bridge" that allows models to apply their strong English-side reasoning capabilities to other languages, even when native training data is scarce.

What Problem Does This Paper Address?

Reinforcement Learning with Verifiable Rewards (RLVR) has recently supercharged the reasoning abilities of LLMs (think DeepSeek-R1). However, RLVR relies on verifiable rewards—tasks where an answer can be objectively checked, like math problems or code execution.

This creates two major bottlenecks:

  1. Domain Gap: In knowledge-intensive fields like medicine or cybersecurity, most data is "unverifiable" raw text (textbooks, blogs, papers) rather than neat Q&A pairs.
  2. Language Gap: While English domain text is abundant, high-quality, verifiable reasoning data in other languages (like Korean, Swahili, or Hindi) is incredibly rare.

Existing methods often rely on simple translation, but this frequently fails because models struggle to map non-English inputs to their internal "reasoning pathways," which are typically dominant in English.

Methods

The authors propose a pipeline that transforms raw English text into a cross-lingual training signal for RLVR.

  1. Verifiable FIM-MCQA Synthesis: They take raw English domain text and use an LLM to extract an educational passage. They then mask a specific span of text and treat it as the "correct answer," generating plausible distractors to create a Fill-In-the-Middle Multiple-Choice Question (FIM-MCQA). Because the answer is a verbatim substring of the original text, it is automatically verifiable.
  2. Code-Switched Task Synthesis: Instead of just translating these tasks, they create "code-switched" variants. This includes:
  3. Option-side Code-Switching: An English passage paired with target-language answer options.
  4. Passage-level Code-Switching: Mixing English and target-language spans within the passage itself.
  5. RLVR Training: They use these synthesized tasks to fine-tune models (Llama 3.2, Qwen 2.5, and SmolLM3) using Direct Alignment from Preference Optimization (DAPO).
The data construction pipeline showing how raw corpora are turned into various monolingual and code-switched training variants.

Datasets

  • Training Source:
    • Cybersecurity: The Primus dataset (500k documents from MITRE, Wikipedia, etc.).
  • Medical: The Biomed-Enriched dataset (450k documents from PubMed).
  • Evaluation Benchmarks:
    • Medical: MedQA, MedMCQA, and native Korean benchmarks like KorMedMCQA-Doctor and KMMLU-Pro-Medical.
  • Cybersecurity: SecEval, CyberMetric, and CTI-MCQ, evaluated across Korean, Swahili, Hindi, and Japanese.

Key Insights and Ideas

  • The "Bridge" Effect: Code-switching acts as a training scaffold. By forcing the model to process English and a target language in the same context to reach a single rewardable answer, the model learns to "bridge" the two languages internally.
  • Input Understanding is the Bottleneck: The authors found that a model's failure in non-English reasoning often starts at the input stage. Code-switched training significantly improves the model's ability to recover English-side semantic meaning from target-language inputs (see Figure 2).
  • Middle-Layer Alignment: Mechanistic analysis showed that code-switched training doesn't just change the output; it actually aligns the model's internal representations in the middle layers (layers 12–24). This suggests the model is learning to map different languages into a shared, language-agnostic "reasoning space."
Analysis showing that better in-domain translation quality (a proxy for input understanding) correlates strongly with better reasoning performance.
Layer-wise analysis showing that the code-switched model (OURS) achieves higher cross-lingual representation alignment in the middle layers compared to baselines.

Key Results

  • Superior Target Performance: On Korean medical benchmarks, the code-switched approach (OURS) consistently outperformed English-only, Target-only, and Bilingual-mixture baselines. For example, using Qwen2.5-3B, the average target-language score improved from 35.01 (Base) to 35.94.
  • Cross-Lingual Consistency: The method significantly reduced "answer asymmetry"—cases where a model gets the English version of a question right but the Korean version wrong.
  • Zero-Shot Transfer: The benefits extended to low-resource languages like Swahili and Hindi in the cybersecurity domain, proving the method's robustness across different scripts and resource levels.
  • No "Tax" on English: Crucially, these gains in target languages came without degrading the model's original English performance.

Limitations and Future Work

  • Compute Intensity: RLVR is expensive. Each training run required roughly 2,500 H100 GPU hours, limiting the authors' ability to test larger models or more exhaustive hyperparameter searches.
  • Evaluation Coverage: While the results are promising, the authors acknowledge that many target-language benchmarks are themselves translated from English, which might introduce artifacts.
  • Language Scope: The study focused heavily on Korean. Future work is needed to see if these "bridges" hold up as effectively for even lower-resource languages or more diverse language families.

Summary updated

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) can improve reasoning in large language models, but specialized non-English domains expose a data mismatch. Raw English domain text is often available, while target-language RLVR tasks with verifiable rewards are scarce. We introduce code-switched task synthesis for cross-lingual RLVR, a data-centric approach that derives target-language and code-switched RLVR task variants from otherwise unverifiable English domain text. Using fill-in-the-middle multiple-choice QA as a seed task format, we synthesize automatically verifiable tasks from domain passages and reshape them into cross-lingual variants. These variants preserve the same evidence-answer relation but change how English and target-language evidence are presented during RLVR. On Korean medical benchmarks and cybersecurity benchmarks across four target languages, code-switched training outperforms matched monolingual and bilingual-mixture baselines on average target-language performance while maintaining comparable English performance. Additional analyses provide evidence of better target-language input understanding and stronger cross-lingual alignment in both model outputs and internal representations. These results suggest that code-switched training helps target-language inputs access domain reasoning learned more strongly in English.
BibTeX
@misc{park2026from,
  author = {Cheonbok Park and Jihwan Kim and Shinyoung Joo and Joosung Lee and Hwiyeol Jo and Jeonghoon Kim and Jaegul Choo},
  title = {{From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis}},
  year = {2026},
  eprint = {2610.25399},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.25399},
}
Figure