Transporting Surgical Robot Skills to Rare Anatomy

Robots that learn surgical sub-tasks, such as picking up a needle or retrieving gauze, are trained on demonstrations recorded in ordinary anatomy. Patients with rare conditions often have anatomy that is ordinary in what the tissue does but unusual in where it sits. In situs inversus the organs are mirror images of their usual positions. A pelvic tumor can crowd an already narrow corridor. Bone can leave less than a centimetre of clearance. These patients are exactly the ones for whom nobody can record hundreds of demonstrations.

We propose to treat rare anatomy as a change of coordinates, not as a new task. Using the patient's preoperative scan, we compute a map from a standard "template" anatomy to the patient's anatomy. Demonstrations, the policy's pretrained prior, and the robot's learned uncertainty model are all carried through that map. Reinforcement learning (RL) fine-tuning, meaning trial-and-error optimization of a pretrained policy against a reward, then only has to correct what the map gets wrong. A safety filter tightens its margins in proportion to how unsure the map is, not in proportion to how unfamiliar the patient's absolute coordinates look.

Core Claim

When a surgical policy moves from typical to atypical anatomy, the shift is mostly in geometry. How a gripper contacts a needle, or how gauze slides, does not change because the liver is on the left. Current methods measure novelty in the robot's raw workspace coordinates, so they mistake a geometric shift for a dynamics shift. They then respond in the wrong way, either by pulling the robot back toward standard paths or by freezing it.

Our claim has two parts. First, moving demonstration guidance into a coordinate frame fixed to the anatomy, and fine-tuning against a prior transported through the patient-to-template map, recovers most of the performance of having demonstrations on the patient. Second, RL fine-tuning is necessary exactly where the map fails to preserve feasibility, which is when the corridor narrows or an obstacle intrudes. A smooth map can preserve the shape of a corridor while shrinking the clearance that the original demonstration paths relied on.

This differs from the closest existing system. SSP wraps a demonstration-guided RL policy in a control barrier function filter. A control barrier function filter is a small optimization run at every control step that minimally edits the commanded action so the robot stays inside a safe region. SSP also shrinks that safe region wherever its learned dynamics model is uncertain. Its no-go zones are fixed geometric primitives, and its uncertainty is computed in absolute task space. Inside a corridor with under 10 mm of clearance, as reported for laser osteotomy by the miniature parallel robot paper, any spurious uncertainty can shrink the safe region to nothing. We change two things: where uncertainty is measured, and what drives the shrinking.

Research Program

Base policy. We begin from demonstration-guided RL on template anatomy using the method of DEX. DEX penalizes the critic for valuing actions far from the demonstrations. It also uses nearest-neighbour lookups in the demonstration buffer to steer the agent back when it drifts. DEX reports high success from as few as 25 demonstrations, which motivates our budget. It also shows the failure we target: when the patient's geometry lies outside the buffer, the nearest neighbours point toward standard paths.

Registration map. For each patient we compute a smooth, invertible registration $\phi$ from patient workspace coordinates to template coordinates, using the preoperative segmentation. Wherever registration is unreliable, we also estimate a per-location uncertainty $\sigma_{\text{reg}}(x)$.

Transported prior. The template-trained prior $\pi_0$ is made to act on the patient through the map, giving a transported prior $\tilde\pi_0$. For a patient state $s$, we evaluate $\pi_0$ at the mapped state $\phi(s)$. Its action is pulled back to patient coordinates through the inverse Jacobian of $\phi$, which is the local stretching and rotation of the map. The DEX nearest-neighbour guidance is computed the same way, in template coordinates.

Fine-tuning objective. Fine-tuning maximizes task reward while staying close to the transported prior:

$$\max_\pi \; \mathbb{E}_\pi\Big[\textstyle\sum_t r(s_t,a_t)\Big] - \beta\, \mathbb{E}_s\, \mathrm{KL}\big(\pi(\cdot\mid s)\,\|\,\tilde\pi_0(\cdot\mid s)\big).$$

Here $r$ is the task reward and $\beta$ sets how strongly the policy is anchored to the prior. Anchoring to a pretrained prior with a KL penalty is the standard way RL fine-tuning in molecular design keeps a policy from collapsing into degenerate shortcuts. We borrow that discipline from that literature, while noting that the domain is entirely different.

Safety filter. We keep SSP's filter structure, but the patient's own segmented structures set the hard constraints. For each structure $O_i$ the barrier is

$$h_i(x) = d(x, O_i) - \big(m_0 + \kappa_r\, \sigma_{\text{reg}}(x) + \kappa_d\, \sigma_{\text{dyn}}(\phi(x))\big).$$

In this expression, $d(x, O_i)$ is the distance from the tool to the structure and $m_0$ is a fixed base margin. The term $\sigma_{\text{dyn}}$ is the dynamics model's uncertainty, evaluated at the mapped state in template coordinates. The constants $\kappa_r$ and $\kappa_d$ weight the two kinds of uncertainty.

The point of this design is that a mirrored or displaced organ no longer looks novel to the dynamics model. Margins therefore widen only where the map itself is uncertain. Tool positions are tracked relative to anatomy seen through the endoscope, not through absolute joint readings. This follows the finding in SRT that up to 5 cm of mechanical slop on the da Vinci Research Kit (dVRK) defeats absolute-coordinate control.

Scope. Phase one uses rigid anatomical walls only. The rigid-body limitation of current fast simulators, noted for Surgical Gym, and the static-scan limitation of the transurethral enucleation framework both apply here. We state them as limits rather than claim to solve deformation.

Benchmark. We will build an atypical-anatomy suite on SurRoL, a dVRK-compatible RL simulator. The tasks are NeedleReach, NeedlePick, GauzeRetrieve and PegTransfer, each placed inside procedurally generated corridors. There are four corridor families:

  1. Template: ordinary anatomy.
  2. Mirrored: inspired by the port and instrument inversions described in the situs inversus liver resection case.
  3. Narrowed: clearances below 10 mm.
  4. Intruding mass: a mass placed in a pelvic-like corridor, motivated by the confined resection sites highlighted in the DARWIN EU chondrosarcoma cohort.

Demonstrations exist only for the template family, 25 per task. Each target family is given between zero and three demonstrations.

Baselines

Do-nothing reference. We run the template-trained DEX policy unchanged on the patient anatomy. This measures how badly the shift hurts.

Strongest existing method. This is SSP wrapped around DEX, as published. It uses absolute-coordinate uncertainty-driven contraction and patient structures as no-go zones. SSP is the only method in the file that jointly handles learned policies and spatial constraints. It reports success on GauzeRetrieve with a cylinder constraint rising "from 12% (unconstrained) to 68%", a 0.00% collision rate on NeedlePick, and a filter latency of 3 to 10 ms per step. We found no reported result on Lacuna for SSP, DEX, or any surgical policy on atypical or rare anatomy. The comparison on the target families is therefore new.

Fairness. All methods get the same 25 template demonstrations and the same budget of target-anatomy simulator steps. Each gets the same hyperparameter search of twenty configurations, tuned on held-out generator seeds of the template family only.

Ablations. Four ablations isolate the key ingredient:

  • Transport only: registration-transported behaviour cloning with no RL.
  • Fine-tuning only: RL fine-tuning anchored to the untransported prior.
  • Transport with absolute uncertainty: transport plus fine-tuning, but with $\sigma_{\text{dyn}}$ computed in absolute coordinates.
  • Margins without registration uncertainty: the full method with the $\sigma_{\text{reg}}$ term removed.

What counts as beating SSP. Constraint violations must be no worse than SSP's. In addition, the method must improve success on the mirrored, narrowed and intruding-mass families by a margin larger than the 95% bootstrap interval across seeds, and must sharply reduce the rate of filter infeasibility or freezing.

Why not just…?

Why not just mirror or warp the demonstrations and run behaviour cloning? This is the most serious threat, and our own transport-only ablation implements it. For pure mirroring, we expect it to come close to the full method. If it does, that is evidence that transport, not RL, is doing the work. Where transport cannot help is when the map shrinks clearance below what the demonstrated paths need. The evidence in SSP that unconstrained imitation policies collide in nearly every attempt under tight constraints suggests that transport alone will violate those constraints. The narrowed and intruding-mass families decide this. If transported cloning matches the full method there too, we will report RL fine-tuning as unnecessary.

Why not just train RL from scratch with anatomy randomized in a fast simulator? Surgical Gym processes "1 million timesteps in just 2.9 seconds", so compute is not the bottleneck. However, SurRoL reports that sparse-reward RL fails without demonstrations. Rare anatomies are also, by definition, the tails that a randomization distribution under-samples. We will run this baseline with demonstrations, with the same simulator steps, randomized over generator parameters that exclude the held-out rare families. If it generalizes to those families anyway, then the per-patient map adds little.

Why not just plan on the preoperative scan with a classical motion planner? For pure reaching inside rigid walls, a planner may well be enough, and we include one on NeedleReach to check. It does not handle the contact-rich tasks. SRT's 0% success for absolute-coordinate execution under hardware slop also suggests that open-loop plans will fail on the physical robot.

Proposed Experiments

Experiment 1: does transport work? We measure success, constraint violation and infeasibility across the four corridor families, with zero to three target demonstrations. The decisive check is whether transport plus fine-tuning matches an oracle DEX trained on 25 demonstrations recorded directly in each target family.

Experiment 2: is SSP's freezing caused by where uncertainty is measured? We test whether moving uncertainty into template coordinates removes the freezing of absolute-coordinate SSP in narrowed corridors. For each episode we record $\sigma_{\text{dyn}}$ in both coordinate systems. The claim is falsified if canonical-coordinate uncertainty is just as high on mirrored anatomy as absolute-coordinate uncertainty.

Experiment 3: are margins calibrated? We corrupt the registration by controlled amounts and test whether violations stay bounded as $\sigma_{\text{reg}}$ grows. The method should become cautious, not unsafe, as the map degrades.

Experiment 4: physical robot. We transfer the best simulated policies to a dVRK operating in 3D-printed template, mirrored and narrowed corridor phantoms. As in SRT, actions are expressed relative to the camera. The comparison against SSP-wrapped DEX uses twenty trials per condition.