Abstract
Force/torque feedback can substantially improve Vision-Language-Action (VLA) models on contact-rich manipulation, but most existing approaches fuse all modalities at a single operating frequency. This design ignores the mismatched sampling rates of real robot sensors, forcing downsampling of the high-frequency contact cues needed for reactive correction. Combined with common VLM-action-expert (AE) pipelines that execute action chunks largely open loop between expensive VLM updates, unified-frequency fusion often yields delayed responses to impacts, stick-slip, and force spikes. We propose FAVLA, a force-adaptive fast-slow VLA that decouples slow perception planning from fast contact-aware control. FAVLA runs a slow VLM at a fixed low frequency to encode modalities to produce latent representations and to predict near-future force variation. A fast AE then executes at a variable high frequency, conditioning on the latest force sequence data to generate reactive actions. We further introduce a force adapter that injects high-frequency force features into multiple AE layers, and adaptively schedules the AE's execution frequency based on the VLM's predicted force variation. Extensive experiments on contact-rich tasks demonstrate that FAVLA significantly outperforms baselines, achieving superior reactivity and success rates, especially with a smaller contact force during manipulation.
Introduction
Current Vision-Language-Action (VLA) models, while proficient in general-purpose reasoning, often falter during contact-rich manipulation tasks where success depends heavily on haptic feedback. The primary bottleneck is a frequency mismatch between sensors: cameras typically operate at 15–30 Hz, whereas force/torque sensors can provide data at 1000 Hz.
Existing models often force all modalities into a single, slow operating frequency, leading to significant information loss and delayed reactions. When executing action chunks in an open-loop fashion, robots cannot respond quickly enough to sudden impacts or resistance, often resulting in mechanical jams or excessive force.
Methodology
FAVLA (Force-Adaptive Fast–Slow VLA) addresses these issues by decoupling high-level semantic reasoning from high-speed reactive control. The architecture consists of three primary components:
- Slow VLM Backbone: Utilizing a large-scale model (PaliGemma), this component operates at a fixed low frequency. It processes vision, language, and historical force data to generate a context (KV Cache). A novel Force Variance Head predicts upcoming force volatility to guide the system's alertness.
- Fast Action Expert (AE): A nimbler transformer that runs at a variable high frequency. It leverages the VLM's context but conditions its outputs on the most recent high-frequency force data.
- Force Adapter: Instead of simple token concatenation, FAVLA employs a cross-attention mechanism to inject high-frequency force features directly into multiple layers of the Action Expert, ensuring haptic feedback remains a primary constraint on the action space.
The system employs Force-Adaptive Inference, where the execution frequency of the Action Expert is dynamically scaled. When the VLM predicts high force volatility—such as an imminent surface contact—the AE frequency increases to ensure rapid reactivity.
Experimental Setup
The model was evaluated on four real-world, contact-rich tasks: USB insertion, gear assembly, box flipping, and board wiping. A custom dataset of 260 trajectories (approximately 1.84 hours) was collected using a Monte robot equipped with X-ARM limbs and 6-axis force/torque sensors. Teleoperation was enhanced by mapping force magnitudes to audio feedback, allowing operators to "feel" the contact during data collection.
Results and Discussion
FAVLA demonstrates significant improvements across all metrics:
- Success Rates: The model achieved an average success rate of 80.8%, substantially higher than the vision-only $\pi_0$ baseline (42.8%) and the state-of-the-art ForceVLA (67.0%).
- Force Compliance: In gear assembly, FAVLA reduced peak contact force from 12.0N (standard $\pi_0$) to 7.7N, indicating safer and more compliant manipulation.
- Reactivity: Ablation studies confirmed that adaptive frequency scheduling provides a superior balance between temporal consistency and responsiveness compared to fixed-frequency alternatives.
Conclusion and Future Work
While FAVLA marks a significant step forward, challenges remain. The model occasionally misinterprets high misalignment forces as successful insertion cues. Future work will explore the integration of tactile "skin" sensors and the scalability of this fast-slow architecture to broader multi-task industrial environments.
Summary updated