No article text was recorded for this version.
reward variance collapse
Version 1 of 170 papers92 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
- Blog: Explore Data Left Behind in Reinforcement Learning for Reasoning Language Models
- Blog: EvolvedGRPO: Unlocking Reasoning in LVLMs via Progressive Instruction Evolution
- Blog: ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking
- Blog: Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training
- Blog: On the Policy Gradient Foundations of Group Relative Policy Optimization: Credit Assignment, Gradient Sparsity, and Rank Collapse
- Blog: GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning
- Blog: Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning
- Blog: Revisual-R1: Advancing Multimodal Reasoning From Optimized Cold Start to Staged Reinforcement Learning
- Blog: From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning
- Blog: Stabilizing Reinforcement Learning for Honesty Alignment in Language Models on Deductive Reasoning
- Blog: Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts
- Blog: No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
- Blog: SFT Overtraining Predicts Rank Inversion via Entropy Collapse Under RLVR
- Blog: EchoRL: Reinforcement Learning via Rollout Echoing
- Blog: Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
- Blog: Exploration Hacking: Can LLMs Learn to Resist RL Training?
- Blog: Toward Honest Language Models for Deductive Reasoning
- Blog: RePO: Replay-Enhanced Policy Optimization
- Blog: ReDit: Reward Dithering for Improved LLM Policy Optimization
- Blog: Discounted Beta–Bernoulli Reward Estimation for Sample-Efficient Reinforcement Learning with Verifiable Rewards
- Blog: DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO
- Blog: NGRPO: Negative-enhanced Group Relative Policy Optimization
- Blog: Learning More with Less: A Dynamic Dual-Level Down-Sampling Framework for Efficient Policy Optimization
- Blog: Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization
- Blog: VideoSSR: Video Self-Supervised Reinforcement Learning
- Blog: XRPO: Pushing the Limits of GRPO with Targeted Exploration and Exploitation
- Blog: EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity
- Blog: Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse
- Blog: COPO: Consistency-Aware Policy Optimization
- Blog: DIVA-GRPO: Enhancing Multimodal Reasoning through Difficulty-Adaptive Variant Advantage
- Blog: Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting
- Blog: Single-stream Policy Optimization
- Blog: Buffer Matters: Unleashing the Power of Off-Policy Reinforcement Learning in Large Language Model Reasoning
- Blog: Train Less, Learn More: Adaptive Efficient Rollout Optimization for Group-Based Reinforcement Learning
- Blog: Embedding-perturbed Exploration Preference Optimization for Flow Models
- Blog: DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
- Blog: Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning
- Blog: Training for Diversity in Image Paragraph Captioning
- Blog: VIDEOP2R: Video Understanding from Perception to Reasoning
- Blog: Frontier Learning: Training LLM Reasoners at the Edge of Capability
- Blog: HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
- Blog: wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models
- Blog: RL with Learnable Textual Feedback: A Bilevel Approach
- Blog: Reinforcement Learning via Self-Distillation
- Blog: Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shuffle
- Blog: R1-ShareVL: Incentivizing Reasoning Capabilities of Multimodal Large Language Models via Share-GRPO
- Blog: Learning to Selectively Learn for Weakly Supervised Paraphrase Generation with Model-based Reinforcement Learning
- Blog: AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting
- Blog: Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning
- Blog: TAPO: Dynamic Teacher and Perturbed Answer Injection for Policy Optimization
- Blog: Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents
- Blog: Knapsack RL: Compute-Efficient Reinforcement Learning via Heterogeneous Rollout Allocation
- Blog: Spectral Policy Optimization: Coloring your Incorrect Reasoning in GRPO
- Blog: PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning
- Blog: DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO
- Blog: G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance
- Blog: Reinforce-Ada: An Adaptive Sampling Framework for Reinforce-Style LLM Training
- Blog: DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Blog: SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
Showing 60 of 70 papers.
History & citation
Current · Permanent link
- This direction split from Overcoming the Zero Variance Problem in Group Reinforcement Learning (81% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_reward_variance_collapse_2c909f35,
title = {reward variance collapse},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/reward-variance-collapse/rdr_2c909f3504c248b8a3d67153d945a3dd},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_reward_variance_collapse_2c909f35_v1,
title = {reward variance collapse},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/reward-variance-collapse/rdr_2c909f3504c248b8a3d67153d945a3dd/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}