WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learning

Xuanlin Jiang, Samuel Hsia, Michael Kuchnik, Zachary DeVito, Minlan Yu, Carole-Jean WuarXiv 2026 · arXiv

TL;DR

WeightBridge is an open-source, highly efficient weight-transfer library designed to eliminate communication bottlenecks in Reinforcement Learning (RL) systems for Large Language Models (LLMs). By automatically discovering weight layout correspondences and executing a topology-aware, redundancy-free transfer plan, it reduces average GPU stall times by up to 42× across diverse parallelization and synchronization configurations.

What Problem Does This Paper Address?

Reinforcement Learning post-training (such as PPO or GRPO) is crucial for boosting LLM capabilities in mathematical reasoning and software engineering. The standard RL workflow alternates between rollout workers (which run inference to generate text trajectories) and trainer workers (which use these trajectories to update model weights). "Weight transfer" is the process of copying these updated weights from trainers back to rollout workers.

As models scale—especially with sparse Mixture-of-Experts (MoE) architectures—and GPU compute outpaces network bandwidth, weight transfer has become a massive bottleneck. In production environments, GPUs can sit idle, stalling on weight transfers for up to 27% of the total execution time.

Optimizing this process is incredibly difficult because training and inference engines are optimized independently. They use entirely different GPU allocations, parallelization strategies (data, tensor, pipeline, or expert parallelism), and specialized runtime tensor formats. This creates complex layout mismatches and communication imbalances across the network. Existing solutions only support narrow slices of this massive configuration space, often suffering from redundant data transfers or single-source bottlenecks.

The standard RL workflow alternating between training and inference engines.
Mismatches and inefficiencies like redundant transfers and imbalanced channels in naive weight transfer.

Methods

The authors introduce WeightBridge, a flexible library that decouples weight transfer from specific backend control planes through four main components:

  1. Automatic Layout Correspondence Extraction: Instead of relying on hardcoded, model-specific rules, WeightBridge treats each engine's load_weights function as a black box. It probes this function using an indexed checkpoint to automatically map which weight elements live on which GPU. To avoid running out of memory from storing 64-bit indices on massive models, it introduces a novel second-order difference (diff2) encoding that drastically compresses the mapping data.
  2. Global Weight Transfer Planner: Once layouts are known, the planner builds a topology-aware, three-stage routing plan: Receive (slicing and routing data), External Exchange (using RDMA across nodes), and Internal Exchange (using ultra-fast NVLink within a node). This ensures no redundant data is sent over the slower cross-node network.
  3. Routing Plan Executor: It breaks the model into chunks and uses a round-based, batched pipeline. This allows the system to completely overlap tensor format conversion kernels with RDMA communication, keeping network links saturated.
  4. Worker Coordinator: It provides a clean, minimal API (connect, send_weights, poll_requests) that natively supports four distinct synchronization modes (synchronous, asynchronous, and staged variants) without modifying the underlying training or inference engines.
The WeightBridge workflow spanning the planning and execution phases.

Datasets

The system was evaluated on real-world workloads using prompts drawn from the DAPO-Math dataset. The evaluation spanned four representative large models: Moonlight-16B-A3B, Qwen3-30B-A3B, Qwen3-235B-A22B, and the 1.03T-parameter Kimi-K2-Instruct mixture-of-experts model.

Key Insights and Ideas

  • Black-Box Probing over Hardcoding: By exploiting the fact that any framework's load_weights function inherently contains the mapping logic, WeightBridge achieves complete generality. It bypasses the need for manual rule-writing or lookup tables for every new model or framework version.
  • Leveraging Hierarchical Networks: Instead of performing a direct all-to-all cross-node transfer, WeightBridge sends exactly one copy of a weight slice to a node via RDMA, and then uses intra-node NVLink to distribute it to other GPUs on that node. This drives communication costs down to the theoretical hardware lower bound.
  • Hiding Format Conversion Overhead: Tensor layout transformations (like transposing or packing for specialized inference kernels) typically introduce massive overhead. WeightBridge batches these operations into a single kernel per round, reducing their cost to less than 3% of total transfer time and completely hiding them behind network communication.

Key Results

  • Massive Stall Reduction: WeightBridge reduces average GPU stall times by up to 42× compared to the state-of-the-art open-source RL framework (Miles).
  • Real Workload Speedups: Across the evaluated models (16B to 1T parameters), WeightBridge achieved speedups between 16.6× and 29.4× over standard broadcast mechanisms.
  • Ease of Integration: To prove its API simplicity, the authors tasked an LLM coding agent (Claude 3 Opus) with integrating WeightBridge. The agent successfully integrated it into Miles in under 30 minutes, achieving a 20× weight-transfer speedup with zero human intervention.

Limitations and Future Work

  • Precision Mismatches: WeightBridge currently assumes that the trainer and rollout engines use the exact same data type precision (e.g., BF16). It does not yet support on-the-fly quantization (e.g., transferring from a BF16 trainer to an FP8 inference engine), which is a common optimization in production.
  • Elasticity and Fault Tolerance: The current implementation relies on a precomputed plan over a fixed, static set of workers. If a node fails or if rollout resources scale dynamically, the plan must be recomputed. The authors note that integrating dynamic relaying patterns (similar to TensorHub or Laminar) is a promising direction for future work.

Summary updated

Abstract

Weight transfer - the propagation of updated parameters from trainers to rollout generators - is becoming an important performance bottleneck in reinforcement learning (RL) systems for LLMs. The central challenge is supporting the diverse trainer and rollout layouts and synchronization requirements of modern RL workloads without sacrificing efficiency. Existing solutions are efficient under some configurations but perform poorly or lack support under others. We present WeightBridge, a flexible, efficient weight-transfer library designed to deliver high performance across diverse RL configurations. WeightBridge first automatically extracts the correspondence between trainer and rollout weight layouts, then plans and executes redundancy-free and load-balanced weight transfer. It exposes a small, general API while coordinating workers across diverse synchronization modes. Across configurations spanning different models, parallelization layouts, and synchronization modes, WeightBridge reduces average GPU stall time by up to 42$\times$ over the state-of-the-art open-source RL framework and achieves high performance in all settings. A coding agent was able to integrate WeightBridge into two different RL frameworks without manual guidance, demonstrating the generality and ease of use of its APIs.
BibTeX
@misc{jiang2026weightbridge,
  author = {Xuanlin Jiang and Samuel Hsia and Michael Kuchnik and Zachary DeVito and Minlan Yu and Carole-Jean Wu},
  title = {{WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learning}},
  year = {2026},
  eprint = {2609.25442},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.25442},
}
Figure