Abstract
Vision-Language-Action (VLA) models represent a significant leap in robotic control, yet their immense parameter counts hinder deployment on resource-constrained edge devices. We present RLRC (Reinforcement Learning-based Recovery for Compressed models), a framework that aggressively prunes VLAs by up to 90% and recovers performance through a synergistic combination of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Our results demonstrate that RLRC-compressed models can outperform their dense counterparts in both seen and unseen tasks while achieving an 8x reduction in memory footprint and a 2.3x increase in inference speed.
Introduction
Vision-Language-Action (VLA) models like OpenVLA have revolutionized robotic manipulation by integrating multimodal understanding with direct motor control. However, these models typically contain billions of parameters, creating a significant "deployment wall" for real-world robotics. High memory requirements necessitate expensive, high-end GPUs, while slow inference speeds prevent the real-time, reactive control essential for dynamic environments. Furthermore, standard compression techniques often lead to a "performance cliff," where the model's physical reasoning capabilities collapse after pruning.
To address these challenges, we introduce RLRC. This three-stage pipeline is designed to shrink massive VLAs while preserving—and in some cases enhancing—their operational success rates on edge hardware.
Methodology
The RLRC pipeline consists of three distinct phases designed to bridge the gap between model size and robotic performance:
- Structured Pruning: Unlike unstructured pruning, which removes individual weights, RLRC removes entire functional blocks or channels from the LLM backbone. This approach is hardware-friendly, enabling direct speedups on standard GPUs. We demonstrate that up to 90% of parameters can be removed while maintaining the underlying architecture's efficiency.
- Performance Recovery: Pruning at high ratios (e.g., 90%) initially compromises model functionality. We recover and enhance performance through a two-step process:
- Supervised Fine-Tuning (SFT): A "warm start" phase using human-annotated data to restore basic task competence.
- Reinforcement Learning (RL): Utilizing Proximal Policy Optimization (PPO) in simulation, the model "practices" tasks. This allows the agent to explore and refine control strategies, often discovering more robust solutions than the original supervised baseline.
- Quantization: For extreme edge deployment, 4-bit post-training quantization can further reduce the memory footprint to approximately 1.7 GB.
Experimental Results
We evaluated RLRC using the ManiSkill3 benchmark, specifically the PutOnPlateInScene25Main suite with an 8-DoF WidowX arm. Testing included 16 In-Distribution (IND) and 9 Out-of-Distribution (OOD) tasks to assess generalization.
Key Findings
- Efficiency Gains: RLRC achieved an 8x reduction in memory usage (from 14.8 GB to 1.8 GB) and a 2.3x boost in inference throughput (from 5.9 to 13.5 samples per second).
- Superior Performance: The RL-recovered model achieved a 90.62% success rate on seen tasks, surpassing the original dense model's 89.06%.
- Robust Generalization: RLRC demonstrated superior generalization on unseen OOD tasks, reaching a 62.5% success rate compared to the original model's 57.81%. This suggests that the exploratory nature of RL helps the model find more robust control strategies.
- The Quantization Paradox: At very high sparsity (90%), 4-bit quantization can actually slow down inference compared to full-precision. This is due to the computational overhead of dequantizing small weight matrices outweighing the benefits of reduced memory bandwidth.
Conclusion
RLRC provides a viable path for deploying state-of-the-art VLA models on resource-constrained robots. By combining structured pruning with a robust SFT and RL recovery phase, we can produce lightweight models that offer high-speed, high-success performance suitable for real-world edge applications. Future work will focus on "robotics-aware" pruning metrics to further optimize spatial reasoning and motor control structures.
Summary updated