ARMOR: Adaptive Curriculum Meta-Learning for Noise-Robust RAG Reasoning

Yan Wang, Yuxin Zhang, Shenyu Zhang, Yongrui Chen, Sheng Bi, Guilin QiarXiv 2026 · arXiv

TL;DR

ARMOR (Adaptive Curriculum Meta-Learning) is a framework that treats noise robustness in Retrieval-Augmented Generation (RAG) as a multi-faceted skill learned through an adaptive curriculum. By employing a "Teacher" scheduler that prioritizes tasks based on learning progress and a novel global advantage estimation, ARMOR enables LLMs to master when to retrieve, how to filter noise, and how to reason through distractions.

What Problem Does This Paper Address?

Retrieval-Augmented Generation (RAG) is the go-to solution for reducing hallucinations in Large Language Models (LLMs). However, RAG has a "noise" problem: retrievers often pull in irrelevant or misleading documents alongside the useful ones.

Current attempts to fix this are usually one-dimensional—focusing only on better filtering or better reasoning—and often fail because these capabilities are interdependent. Furthermore, training models to be robust via Reinforcement Learning (RL) is notoriously difficult. Reward signals for different tasks (like "knowing when to stop searching" vs. "ignoring a distractor") can interfere with each other, leading to optimization instability or "mode collapse," where the model gets stuck in a repetitive, suboptimal behavior.

Methods

The authors propose ARMOR, which decomposes noise robustness into three sub-tasks:

  1. Knowledge Boundary Perception: Knowing if the model's internal knowledge is enough or if it needs to retrieve.
  2. Retrieval Verification: Identifying which retrieved documents are "golden," "inconsequential," or "irrelevant."
  3. Robust Reasoning: Generating the correct answer even when the context is cluttered with noise.
Figure 1: The overall framework of ARMOR. (Left) The Adaptive Task Scheduler functions as a Teacher, dynamically prioritizing one of the three robustness sub-tasks (Knowledge Boundary Perception, Retrieval Verification, and Robust Reasoning) for each training batch based on the Absolute Learning Progress (ALP). (Right) The RAG Generator (Student) generates multiple rollouts to learn the selected task. (Bottom Right) Inter-Group Advantage Estimation computes reward advantages across the global batch scope rather than within groups, effectively mitigating mode collapse and stabilizing the decoupled training process.

Key Innovations:

  • Adaptive Task Scheduler (The Teacher): Inspired by human pedagogy, this component uses Absolute Learning Progress (ALP) to decide which sub-task the model should focus on in each training round. It prioritizes tasks where the model shows the highest potential for improvement.
  • Inter-Group Advantage Estimation: Standard RL algorithms like GRPO calculate rewards by comparing different outputs for the same question. However, in RAG sub-tasks (like a True/False search decision), the output space is so small that all rollouts often look identical, making it impossible to calculate a relative advantage. ARMOR solves this by normalizing advantages across the entire training batch, providing a much more stable and diverse training signal.
  • Structured Warm-up: Before RL, the model undergoes Supervised Fine-Tuning (SFT) on "reasoning chains" to learn the basic "retrieve-verify-reason" workflow.

Datasets

The researchers evaluated ARMOR on three complex multi-hop reasoning datasets:

  • 2WikiMultiHopQA
  • MusiQue
  • HotPotQA (used as an out-of-distribution test set to check generalization).

They reconstructed these datasets into structured reasoning chains, where each step includes an intermediate thought, a sub-question, a retrieval decision, and a verification tag for the retrieved documents.

Key Insights and Ideas

  • Sequential Learning is Essential: The paper demonstrates that a rigid, fixed training order leads to "catastrophic forgetting." The adaptive scheduler is crucial because it balances exploration and exploitation, ensuring the model doesn't move on until it has actually made progress.
  • Global vs. Local Normalization: In tasks with constrained outputs (like classification tags), intra-group comparison is a recipe for mode collapse. Expanding the scope to the batch level (Inter-Group) is a simple but highly effective fix for RL stability.
  • The "Rejection" Mechanism: One of the most interesting findings is that ARMOR learns a "rejection" mechanism. Even at 100% noise (where all retrieved docs are irrelevant), ARMOR often matches or beats a model that didn't retrieve anything at all, proving it has learned to suppress irrelevant context rather than being misled by it.

Key Results

ARMOR achieved state-of-the-art (SOTA) performance across all benchmarks, significantly outperforming strong baselines like IRCoT and RAG-RL.

  • Robustness to Extreme Noise: In controlled experiments, ARMOR maintained a correctness score of 69% even when the noise ratio was as high as 80%. Baselines like IRCoT and Search-R1 saw precipitous performance drops under the same conditions.
  • Efficiency on Small Models: ARMOR is particularly effective for smaller models. On Qwen-3-1.7B, it improved average correctness by 38.8% over the strongest baseline.
  • Dynamic Focus: Training logs show the scheduler's adaptability. For example, it might focus heavily on "Knowledge Boundary" early on and shift to "Retrieval Verification" once the model knows when to search but not yet how to filter.
Figure 2: Robustness analysis under controlled noise ratios (from 0% to 100%) with fixed retrieval size of K = 5.

Limitations and Future Work

  • Domain Specificity: The study focused on general-domain multi-hop QA. The authors acknowledge that high-stakes fields like medicine or law might require different constraints and structural complexities.
  • Language Scope: The current experiments are limited to English-only corpora.
  • Data Preparation: ARMOR relies on a structured warm-up phase, which requires high-quality, reconstructed reasoning chains—a potentially expensive data engineering task for new domains.

Future work could involve extending ARMOR to cross-lingual settings and exploring its performance in specialized technical domains.

Summary updated

Abstract

Retrieval-Augmented Generation (RAG) systems have demonstrated remarkable effectiveness in mitigating hallucinations by incorporating external knowledge. However, the retrieval process inevitably introduces noise, posing significant challenges to RAG robustness. Fundamentally, noise robustness is a composite competency encompassing noise identification, retrieval filtering, and robust reasoning. Unfortunately, existing methods often fail to capture this holistic nature, typically restricted to single-dimensional enhancements or hampered by noise-induced optimization interference. To address these limitations, we decompose RAG robustness into three sub-tasks: knowledge boundary perception, retrieval verification, and robust reasoning. Accordingly, we propose ARMOR, an Adaptive Curriculum Meta-Learning for Noise Robust RAG Reasoning. Inspired by the teaching process, ARMOR introduces an adaptive task scheduler as a teacher to curate training plans according to the absolute learning progress of RAG generators (Student). Distinct from GRPO, ARMOR utilizes inter-group advantage estimation during learning to effectively mitigate mode collapse caused by limited output spaces. Experiments on three complex reasoning datasets (2WikiMultiHopQA, MusiQue, and HotPotQA) indicate that ARMOR achieves synergistic improvements in retrieval noise robustness, significantly outperforming baselines in output quality.
BibTeX
@misc{wang2026armor,
  author = {Yan Wang and Yuxin Zhang and Shenyu Zhang and Yongrui Chen and Sheng Bi and Guilin Qi},
  title = {{ARMOR: Adaptive Curriculum Meta-Learning for Noise-Robust RAG Reasoning}},
  year = {2026},
  eprint = {2611.11771},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11771},
}
Figure