Robots that learn by trial and error on real hardware usually sit behind a safety filter: a small optimizer that checks each commanded action and, if needed, nudges it so the robot stays out of forbidden regions. These filters assume that a command changes the robot's motion right away. Cable-driven surgical robots break that assumption. When a motor reverses direction, the slack cable or gear gap has to be taken up before the tool moves at all, and the tool's true position differs from what the joint encoders report by an amount that depends on which way the motor last turned. On the da Vinci Research Kit (dVRK), this "slop" produces end-effector errors of up to 5 cm (SRT). A safety filter that ignores it is not certifying the real robot.
We propose to give the safety filter an explicit, cheaply estimated backlash state: roughly, where the motor sits inside the gap and which side of the gap is engaged. That variable is estimated from command history and, where available, motor current. The filter then shifts its safety margin by the known direction-dependent offset and widens it only while the transmission is inside the gap. The claim is that this recovers near-zero constraint violations during online learning on backlash-affected hardware, and does so with much less conservatism than a worst-case margin. A second payoff follows: once safety is enforced as a hard filter that is correct for the physical robot, there is no need for hand-tuned collision and jerk penalties in the reward. The learning signal can be a sparse task-success reward on top of a demonstration-trained policy.
Core Claim
Current robust barrier filters on the dVRK treat modeling error as memoryless noise around a model of the form $\dot x = f(x,u)$. The clearest example is SSP, which learns a Neural ODE (a neural network that predicts continuous-time dynamics) with an uncertainty estimate and shrinks the safe set as that uncertainty grows. Backlash error is not noise. It is mostly a deterministic offset whose sign is set by the history of direction reversals, plus a short window of lost control authority while the gap is traversed. A state made only of joint positions and velocities is therefore not Markov: identical encoder readings can correspond to tool positions a full gap-width apart. A memoryless uncertainty model can only cover this by inflating its bound everywhere. We model the tool position as
$$p = g(q) + d(z) + e, \qquad \|e\| \le \varepsilon(z),$$
where $q$ is the encoder reading, $g$ is nominal forward kinematics, $z$ is the backlash state (position within the gap and engaged side), $d(z)$ is the direction-dependent offset, and $\varepsilon(z)$ is a residual bound. The bound is small when the drive is engaged and roughly the gap width while $z$ is in transit. The filter enforces a discrete-time barrier condition on
$$\tilde h = h\big(g(q)+d(z)\big) - L_h\,\varepsilon(z),$$
where $h$ is the signed distance to the forbidden region and $L_h$ is its Lipschitz constant. It also propagates $z$ forward so that it knows when a braking command will actually take effect. The new element is this hybrid state and the direction-dependent margin. The QP machinery for the filter is reused from SSP, and the learning algorithm is reused from existing constrained RL.
Research Program
The work has three pieces.
Backlash state estimator. From commanded motor positions we run a standard play-operator hysteresis model, which tracks the gap position given the reversal history. Where motor current is available, we correct the model using the signature documented by the tendon-sheath current study. That study found that a flat current plateau marks dead-zone traversal and a sharp rise marks re-engagement, and it used this to cut single-axis RMSE by 71–76% with an open-loop piecewise model. The same study reports that gap width drifts as the tendon path curves. We therefore re-estimate gap width online from the current transitions instead of fixing it from a table. This is where our approach departs from the static calibration of the two-phase debridement pipeline, which needs a human-supervised grid step.
Backlash-aware filter. The filter is a QP around any nominal policy, following SSP's form. It uses $\tilde h$ in place of $h$. When $z$ indicates that a reversal is needed to back away from a boundary, it adds a lookahead constraint so that the braking command is issued early enough for the gap to close before the boundary is reached.
Safe improvement without shaped rewards. The nominal policy is a demonstration-trained policy using the hybrid-relative actions of SRT, which are robust to absolute slop. A residual policy is trained with a constrained RL method: PPO with a PID-damped Lagrange multiplier, from Safety Gymnasium. The reward is sparse success only, and the constraint is no-go-zone cost only. The filter runs throughout training, so violations are prevented during exploration and not merely at convergence.
This design targets two known weaknesses. The first is the "safe but useless" standstill that strict guarantees can cause (Quanyi Li et al.), which we attack by avoiding a global worst-case margin. The second is the risk that feedback across a deadband chatters (Ficht et al.). Because $z$ is part of the filter state, we can measure whether the filter itself induces reversals.
Simulation uses SurRoL, which runs in position-control space without drivetrain hysteresis. We will add an actuator layer to it with per-joint deadband, cable elasticity, and a synthetic current channel. The layer's gap widths will be swept up to the dVRK slop magnitudes reported by SRT. Hardware validation uses a dVRK with an external camera to measure the true tool position, so that violations are counted from real tool positions and not from encoder-reported ones.
Baselines
The naive reference point is the demonstration-trained hybrid-relative policy run with no filter and no online learning. SRT reports 90–100% success for this policy on its dVRK tasks. It gives the success level we must not lose and the violation rate we must beat once no-go zones are added. It will be trained on the same demonstrations, with the same action space, as our nominal policy.
The strongest existing method is SSP: a Neural ODE dynamics model with a robust CBF-QP wrapped around a demonstration-derived policy. SSP reports a 0.00% collision rate on SurRoL NeedlePick. On GauzeRetrieve with a cylinder constraint, it raised success from 12% (unconstrained) to 68%. Those numbers were obtained without drivetrain backlash. We will retrain SSP's Neural ODE on data from our backlash-augmented simulator, so it sees exactly the same transitions and the same data budget as our estimator, and give it the same QP solver and the same hyperparameter search budget. Our method beats SSP if, at dVRK-scale gap widths, it keeps collision rate at or near 0% where SSP's rises, while matching or exceeding SSP's 68% GauzeRetrieve success. We will also report SSP at zero backlash to confirm our reimplementation reproduces its reported numbers.
The constant-margin robust filter is the direct competitor to our central idea. It is a standard CBF whose margin is inflated by the worst-case gap everywhere. We beat it only if we get equal violation rates with higher success, shorter completion time, and smaller average clearance to boundaries.
A compensate-then-filter baseline linearizes the actuator first, using the current-based piecewise inverse from the tendon-sheath study (or the gap model inverted feedforward), and then applies a standard backlash-free CBF. Standard constrained RL without a filter covers the training-time guarantee question: CPPO-PID from Safety Gymnasium, FOCOPS, and the cost-so-far method of Backward Value Functions. A hand-shaped PPO baseline completes the set, with collision and jerk penalties tuned over a grid equal in total compute to our full run.
Ablations isolate the key ingredient. We replace $z$ with an oracle, with history-only estimates, and with current-corrected estimates. We also give SSP's Neural ODE a reversal-history input, which tests whether "just add history features" to a memoryless model is enough.
Why not just…?
Why not just inflate the barrier margin by the worst-case backlash? This is the honest competitor. It is sound, but the margin would be on the order of the centimeter-scale slop SRT documents, and surgical workspaces around tissue and needles are much tighter than that. We expect it to be safe and largely useless. The constant-margin baseline measures this directly, and if it matches our success rate at realistic gap widths, the core claim fails.
Why not just compensate the backlash first and use an ordinary filter? Current-based compensation removes most but not all error; 2-DOF RMSE drops by only 58% in the tendon-sheath study. The same study reports parameter drift with path curvature and jerk at segment boundaries. Residual error during reversals is exactly where boundary crossings happen. Compensation is complementary, since our $d(z)$ is a compensator, so the real question is whether the direction-dependent residual bound $\varepsilon(z)$ adds safety beyond it. The compensate-then-filter comparison settles that.
Why not just rely on relative actions and visual servoing, as SRT and intermittent visual servoing do? These methods correct steady-state error well. They offer no constraint on unobserved or occluded regions, and camera latency makes them react after the tool has already moved. We keep SRT's action space for exactly its strengths and add only what it lacks, a filter. The unfiltered-policy baseline shows how often such policies cross boundaries once no-go zones are added.
Why not just use SSP's uncertainty-aware Neural ODE, which should learn the slop? A model conditioned only on instantaneous state cannot tell the two engaged sides of the gap apart. It must widen its uncertainty everywhere, so it ends up either conservative or unsafe. The history-augmented SSP ablation tests whether adding reversal history recovers our result. If it does, our contribution narrows to the estimator and the lookahead constraint, and we will say so.
Proposed Experiments
The first decisive check is a simulation stress test. On backlash-augmented SurRoL NeedlePick and GauzeRetrieve with SSP-style no-go zones, we sweep gap width from zero to dVRK scale. For every baseline we plot collision rate against the real tool position and task success. The prediction is that SSP stays near its reported 0% at zero backlash, but its violations concentrate at direction reversals as the gap grows. Our filter should stay near zero with success well above the constant-margin filter.
The second check is training-time safety. We count cumulative violations over the entire residual-RL run, compared against CPPO-PID, FOCOPS, and BVF. We report final success, and we compare against the hand-shaped PPO grid to test whether sparse reward plus a correct filter matches the best tuned penalty setting without any tuning.
The third check is chattering. We measure filter-induced direction reversals per second and actuator current variance, which tests the concern raised by Ficht et al. and whether CPPO-PID's multiplier damping interacts with physical limit cycles.
The fourth check is on hardware. On a physical dVRK with phantom tissue and camera-measured ground truth, we run a small number of residual-learning sessions for our filter and for SSP. We report violations per hour counted from true tool positions, and estimator error of $z$ against visually measured reversal offsets.
The idea fails if the constant-margin or compensate-then-filter baseline matches us on both safety and success at realistic gap widths. It also fails if the history-augmented Neural ODE does, in which case the backlash state is not buying anything a generic learned model cannot.
References
- Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks
- SSP: Safety-guaranteed Surgical Policy via Joint Optimization of Behavioral and Spatial Constraints
- Non-Linear Hysteresis Compensation of a Tendon-Sheath-Driven Robotic Manipulator Using Motor Current
- Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure
- Safety Gymnasium: A Unified Safe Reinforcement Learning Benchmark
- MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning
- Direct Centroidal Control for Balanced Humanoid Locomotion
- SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning
- First Order Constrained Optimization in Policy Space
- Constrained Markov Decision Processes via Backward Value Functions
- Intermittent Visual Servoing: Efficiently Learning Policies Robust to Instrument Changes for High-precision Surgical Manipulation