Where May a Catheter Push? Region-Specific Force Budgets for Autonomous Endovascular Navigation

Reinforcement learning agents can already steer a catheter through simulated aortic arches into branch arteries faster and along shorter paths than an experienced surgeon. They do it by pressing harder on the vessel walls. This proposal argues that the fix is not a single, stronger "be gentle" penalty, because pressure is not uniformly bad. Interventionalists routinely brace a catheter against the outer curve of the aortic arch to gain the leverage needed to enter a sharply angled branch. The same force at a branch opening, or applied by the tip inside a narrow branch, is what causes injury. We propose to give the learning agent a separate force allowance for each anatomical region of the vessel. Each allowance is calibrated to how much force human experts apply in that region, and a learned multiplier enforces it during training. The agent can then spend force where bracing is useful and is held tightly where contact is dangerous.

The claim is specific and falsifiable. On hard anatomies, a single global force constraint must choose between blocking the brace an agent needs and permitting damaging contact elsewhere. Region-specific budgets avoid that choice. If this is right, region-budgeted agents should match the human expert's per-region force profile while keeping most of the agent's speed advantage, and a global constraint should not. The effect should be largest on the anatomies where current agents struggle most. Contact force is used only to shape training and is never needed as a policy input, so the resulting policy does not depend on force sensors that clinical labs lack.

Background and the Gap

The platform is CathSim. CathSim: An Open-Source Simulator for Endovascular Intervention builds a MuJoCo simulation of a follower robot that pushes and rotates a catheter, modeled as a chain of rigid links, through aortic arch meshes scanned from silicone phantoms. It reports wall contact forces whose distribution it validated against benchtop load-cell measurements. It also benchmarks PPO and SAC and finds that "the Type-I aortic arch presented more challenges than the Type-II, with the LCCA target being more difficult to reach than the BCA." Here BCA is the brachiocephalic artery and LCCA the left common carotid artery.

Autonomous Catheterization with Open-source Simulator and Expert Trajectory trains an Expert Navigation Network (ENN), a SAC agent that sees both images and the robot's joint states, across four arch shapes. It reaches 100% success on both BCA and LCCA. Its BCA path is 15.8 cm against 28.8 cm for a human surgeon, and it finishes in about a quarter of the human's time. The same paper reports that the human applied systematically lower wall forces and calls for safety-aware rewards. That paper motivates this proposal: it found the gap, but it did not propose an objective that closes it. The review in Advancing Embodied Intelligence in Robotic-Assisted Endovascular Procedures adds that existing endovascular agents rely on simple scalar rewards, and that internal force and joint signals are not available in real catheterization labs.

The device literature supplies the central intuition. Several studies find that a uniform, symmetric design fails on asymmetric anatomy and that the remedy is to decouple requirements by region:

  • Tailored funnel stents for gastric outlet obstruction spread radial force to match the shape of the obstruction (Ding Shi et al.).
  • The LAmbre occluder separates a small internal anchor from a large external cover (Zhang et al.).
  • Pre-shaped standard funnels approximate individualized designs well (big funnel vs. individualized stents). This is a useful warning that a small set of standard regions may suffice and full patient-specific tailoring may not be needed.

Our analogue separates "where force may be used to anchor" from "where force must be minimal."

The reward-design literature constrains the method. Redundancy Resolution as Action Bias shows that added penalties interfere with the main task: weighted too heavily they block it, weighted too lightly they are ignored. Its own remedy, steering in the robot's redundant directions that leave the task unaffected, does not apply here. A catheter's whole body touches the wall, so no movement leaves contact unchanged.

Multiplicative rewards, as in SkillMimic and PMP, multiply all sub-rewards together so the total collapses to zero if any single requirement fails. They prevent the agent from neglecting one goal, but they give no learning signal early in long tasks. PMP needed injected demonstrations to get training started. Fast RL supplies the method we reuse. It adapts the weights on several objectives with mirror descent, a weight-update rule that changes each weight in proportion to how badly its objective is doing, so as to raise the worst-performing objective. It has been tested only on language-model alignment. Offline Pareto search such as MO-GAMBIT maps the trade-offs between objectives for static catheter placement. It takes up to 48 hours and does not produce a feedback controller.

Core Mechanism

Each arch mesh is divided into $K$ regions using its centerline. Candidate regions are the outer arch wall, the inner arch wall, a neighborhood around each branch opening, the branch lumen, and the descending aorta. Every contact point is assigned to the region of its nearest centerline segment and wall side. Contacts are further tagged as tip contacts or shaft contacts, because tip forces are the clinically dangerous ones.

Let $F_k(\tau)$ be the 95th-percentile contact force in region $k$ over trajectory $\tau$, and let $b_k$ be that region's budget. We solve

$$\max_\pi \; \mathbb{E}_\pi[R_{\text{nav}}] \quad \text{s.t.} \quad \mathbb{E}_\pi[F_k(\tau)] \le b_k \;\; \text{for } k=1,\dots,K,$$

where $R_{\text{nav}}$ is the ENN-style navigation reward for progress and success. Each region has a multiplier $\lambda_k \ge 0$. The multipliers are updated by an exponentiated (mirror-descent) step on that region's normalized violation, as in Fast RL:

$$\lambda_k \leftarrow \lambda_k \exp\big(\eta\,(\hat F_k - b_k)/b_k\big),$$

where $\hat F_k$ is the batch estimate of the region's force and $\eta$ is a step size. The constraint therefore tightens only in the regions currently over budget. Unlike a product of sub-rewards, the navigation reward stays dense throughout training.

Budgets are not hand-tuned. We set $b_k$ from the per-region force distribution of the human expert trajectories recorded in CathSim by the ENN authors. We also study budgets at one half and one quarter of the human values. Because regions are defined geometrically, budgets carry over automatically to an arch the agent has never seen. The policy also receives a short morphology descriptor, consisting of take-off angles, arch type, and branch radii, so it can tell which regions it is likely to face.

Research Program

The work has three parts.

Instrumentation. We first add region-tagged force logging to CathSim. Using the human trajectories, we then test whether expert force is genuinely heterogeneous across regions, for example high on the outer arch wall and low at branch openings. If expert forces turn out to be roughly uniform across regions, the premise fails and we stop. That check is cheap and comes first.

Training. We then train SAC and PPO agents, both of which CathSim benchmarks, under each objective described below. We use the ENN's image-plus-joint-state inputs. We also train image-only variants to show that the policy needs no force input.

Analysis. Finally, we examine where each agent's force goes, region by region. We trace the trade-off curve between path length and per-region force by sweeping a single scale factor on all budgets. We test transfer by training on three arch shapes and evaluating on the held-out fourth.

Baselines

The naive reference is the ENN agent itself: SAC with the navigation reward and no force term, retrained in our setup with the authors' inputs. The cited paper reports 100% success on BCA and LCCA and a 15.8 cm BCA path, against 28.8 cm for the human. For wall-force magnitudes I found only the qualitative statement that humans used lower force; there is no reported result found on Lacuna for the ENN's contact-force values. We will therefore measure them ourselves, and they become the "safety gap" figures to beat.

The strongest competitor is a single global force constraint, enforced with one multiplier using the same mirror-descent update. Its budget is set to the human's 95th-percentile force over the whole trajectory. This is the obvious way to act on the ENN authors' call for gentler rewards, and it differs from our method only in having one region instead of $K$. A second strong competitor is a linear penalty $R_{\text{nav}} - w\,F$, with $w$ swept over a logarithmic grid. We report the best point on the trade-off curve rather than one hand-picked weight.

The ablations isolate the key ingredient. In the first, per-region weights are fixed at their final learned values and held constant throughout training, which tests whether adaptivity matters. In the second, regions are randomly relabeled while keeping the same number of multipliers, which tests whether the benefit comes from anatomy or merely from having more knobs. In the third, the tip/shaft distinction is removed.

All methods get the same environment steps (matched to the ENN training budget), the same network, five seeds, and an equal hyperparameter search budget of about 20 trials per method.

Beating the strongest competitor means three things on Type-I LCCA, the hardest case:

  • reaching the success rate of the global-constraint agent or better at matched or lower branch-opening tip force;
  • doing so with a statistically indistinguishable or shorter path;
  • showing per-region force distributions that a Mann-Whitney test, the test CathSim used for its own validation, cannot distinguish from the human's in the dangerous regions.

Why not just…?

Why not just tune a weighted sum of navigation reward and force penalty? The redundancy-resolution paper documents exactly the failure we expect: heavy weights stall progress and light weights are ignored. The ENN result is the light-weight extreme. Still, a well-swept linear penalty may do fine on easy BCA targets in Type-II arches, so we sweep it fully and report its whole trade-off curve. Our claim concerns hard anatomies, where any single weight must over-penalize the useful brace. If its curve dominates ours on Type-I LCCA, the claim is refuted.

Why not just a single global force constraint? This is the real threat, and it may work as well wherever bracing is unnecessary. The decisive test is the per-region force breakdown on Type-I LCCA. If agents under a global budget achieve success only by exceeding human tip force at the branch opening, or reach low force only by losing success, then regionalization is doing work. If the two methods tie everywhere, we will report that the simpler method suffices. The randomized-region ablation separates anatomical structure from extra capacity.

Why not just imitate the human expert, who already uses less force? The ENN data show the human path is about 1.8 times longer and takes roughly four times as long. Cloning the human would inherit that, including longer fluoroscopy exposure. Imitation also covers few demonstrations and generalizes poorly outside them, as Imitation Is Not Enough found in driving. Our method uses the human data more selectively, as per-region force budgets rather than as trajectories to copy. We include a behavioral-cloning agent on the same trajectories so readers can see how much speed is gained at equal force.

Decisive Checks and Limits

The first check, run before any training, is whether human force is heterogeneous by region. The second is the Type-I LCCA comparison against the global constraint. The third is held-out-arch transfer, where region budgets must carry over without retuning. As a crude robustness probe, we vary MuJoCo's contact softness parameters. This is not a model of compliant, pulsating vessels, and every result inherits CathSim's rigid-wall assumption. The proposal therefore claims better force allocation in simulation, not clinical safety.