Anticipating Catheter Failure by Valuing Each Push in Its Anatomical Context

A robot steering a catheter through the aorta makes hundreds of tiny decisions: push a millimetre, twist a few degrees, pull back. Whether a given push is good or dangerous depends entirely on where the tip is. In a straight segment, pushing is progress. At an acute branch, the same push can make the catheter body buckle and slide back out of the target vessel. Clinicians call this herniation, and it shows up several steps after the push that caused it. Sports analytics has spent a decade on the same structural problem: scoring each individual action by how much it changes the probability of the next decisive event, conditioned on the exact game situation. We propose to import that idea into robotic catheterization. The model would score each actuation step by how much it moves two probabilities: that the next decisive event is a successful cannulation, and that it is a failure (herniation or excessive wall force). The state is described in local anatomical terms, not in terms of one particular patient's vessel map.

The practical payoff is a failure anticipator that can be tested on anatomies it has never seen. It is used in two ways: as a filter that vetoes risky actions of an existing navigation policy, and as a diagnostic that attributes failures to the specific earlier step that caused them. The central bet is that failure risk, expressed in a local anatomical frame, transfers across vessel shapes far better than the navigation policy itself.

Core Claim

Reinforcement learning (RL), which trains a controller by trial and error against a reward, already navigates simulated aortic arches well. In CathSim, proximal policy optimization (PPO) reaches the brachiocephalic artery in a Type-I arch 97% of the time from stacked image observations (CathSim: An Open-source Simulator for Autonomous Cannulation). The same authors report that success depends strongly on arch type and target artery (CathSim: An Open-Source Simulator for Endovascular Intervention). These are policies trained and tested on the anatomy they were built for.

The clinical literature says the hard cases are the anatomically unusual ones. A case report on angiography through an aberrant right subclavian artery describes succeeding only by forming the Simmons curve (a hook-shaped catheter loop) at the aberrant origin, then twisting slowly and retracting, because forward pushing herniated the catheter into the descending aorta (Successful right transbrachial cerebral angiography in a patient with aberrant right subclavian artery). Aneurysm series report conversion to open surgery when branch angles defeat catheter delivery (Su et al., Chen et al.).

Our claim has three parts. First, a model can be trained on off-policy rollouts from many geometries to predict, for a state and a candidate action, the change in probability that the next decisive event is a failure. Second, if that model sees only local anatomical descriptors, its predictions transfer to held-out and deliberately distorted anatomies. Third, used as an action filter, it reduces herniation and force violations of a fixed navigation policy on those anatomies more than a reactive force threshold or the policy's own critic, and it warns earlier.

The sports framing matters for a specific reason. The hockey work models the probability of which team scores next, with separate outcomes for both sides (Routley et al., Schulte et al., Schulte). That structure keeps "risk of disaster" as its own quantity rather than folding it into one scalar reward. Offense-only valuation schemes like STARSS (Decroos et al.) are criticized for under-representing exactly that side. In catheterization, the failure side is the clinically important one.

Research Program

We define a state $s$ in a local frame attached to the catheter tip. It contains the centerline curvature at the tip and over the next few millimetres, the angle between the current lumen and the target branch, lumen diameter, and the tip's heading relative to the centerline. It also includes the normal and friction contact forces along the catheter chain, which CathSim exposes and validates against load cells (Mann-Whitney $p \approx 0.445$). Finally, it includes "slack": inserted length minus tip progress along the target centerline, a direct precursor of buckling.

We define herniation operationally. It occurs when inserted length rises while tip progress along the target branch falls over a short window, or when the tip exits the target branch. A force violation is any contact force above a fixed limit. Both are failure events, and cannulation is the success event.

We learn two action-conditioned heads. The first is $Q_{\text{fail}}(s,a)$, the probability that the next decisive event within a horizon $H$ is a failure after taking action $a$ in state $s$. The second, $Q_{\text{succ}}(s,a)$, is defined the same way for success. Their state-only counterparts $V_{\text{fail}}(s)$ and $V_{\text{succ}}(s)$ average over the behavior data. The quantity we care about is the failure impact of an action, mirroring the hockey value delta:

$$\text{Impact}_{\text{fail}}(s,a) = Q_{\text{fail}}(s,a) - V_{\text{fail}}(s),$$

which is positive when $a$ makes a failure more likely than the typical action in that situation.

Both heads are trained by temporal-difference learning on rollouts, meaning each prediction is bootstrapped from the prediction at the next state. The rollouts come from a deliberately mixed set of behavior policies: PPO checkpoints at several training stages, scripted clinician-style maneuvers (push, twist-and-retract, loop formation), and randomized actions. They are collected across a family of geometries. Following the warning in MERLION that value estimates become overconfident on actions and states outside the data, we add a conservative penalty on actions poorly covered by the data. We treat this penalty as a component to ablate, not a contribution.

The geometry family is an engineering contribution of its own. CathSim ships Type-I and Type-II arch meshes. We generate parametric variants by bending supra-aortic branch origins to steeper take-off angles and by adding an aberrant-subclavian-like origin distal to the left subclavian. We then split variants into training and held-out sets by branch angle, so that test anatomies are strictly more acute than anything seen in training.

At deployment, the anticipator filters a fixed navigation policy. If the policy's proposed action has $\text{Impact}_{\text{fail}}$ above a threshold $\tau$, the filter substitutes the lowest-impact action from a small candidate set: the proposal, retraction, rotation in either direction, and hold. The threshold $\tau$ is chosen on training anatomies only.

We deliberately do not build relational action graphs as in AGILE or meta-learned macro-action credit sharing as in MASP. The catheter's action space here is two-dimensional (insertion and rotation). A candidate set of a handful of actions makes the interdependence those methods target small enough to enumerate. Adding them would dilute the test of the one ingredient we care about.

Baselines

The naive reference is the navigation policy with no filter: PPO trained on the training geometries with CathSim's standard progress reward and contact-force penalty, deployed as-is on held-out geometries. Its in-distribution anchor is the reported 97% Type-I brachiocephalic success (CathSim arXiv). We expect, but have not verified, lower success and frequent herniation on steep variants. No reported result was found on Lacuna for herniation rates in CathSim or for success on geometries outside Type-I and Type-II.

The first strong competitor is a reactive force-threshold filter. It vetoes any action taken while the measured contact force exceeds a limit, and also any action taken while slack exceeds a limit. It uses the same candidate replacement set and thresholds tuned on the same training anatomies. It is the strongest simple competitor because it uses the same privileged force and slack signals our model sees, without any learning.

The second is a critic-based filter. It uses PPO's own value function, with its advantage estimate standing in for impact. Its purpose is to test whether a separate failure probability adds anything over the scalar critic the policy already has.

The third is domain-randomized PPO, retrained over the full set of training geometries with the same environment-step budget as the data used to train our anticipator plus its base policy. This tests whether simply training the policy on more anatomies closes the gap.

The key ablations are three. The first feeds the anticipator a global state (tip position in world coordinates plus stacked images, as in CathSim's image observations) instead of the local anatomical frame; this isolates the transfer claim. The second collapses the two heads into one scalar value with a failure penalty; this isolates the benefit of separate event probabilities. The third removes the conservative penalty.

All methods share the same simulator, the same held-out geometry split, the same candidate action set, and an equal threshold-tuning budget on training anatomies. Our method beats a baseline if it cuts failure events per episode on held-out geometries by a margin that is significant across seeds and geometries, and does so without reducing cannulation success.

Why not just…?

Why not just threshold the contact force and the slack? This is the most serious threat, because it reads the same signals our model reads. Our argument is timing: a threshold fires when danger is already present, whereas herniation starts with pushes that load the catheter body without yet exceeding any limit. The experiment settles it by measuring warning lead time (steps between first alarm and failure) and failure reduction. If the threshold filter matches us on held-out geometries, the learned value adds nothing and we will report that.

Why not just use the policy's critic, since the value delta is already PPO's advantage? Formally, $V(s')-V(s)$ plus reward is the temporal-difference error PPO already computes. The difference we are testing lies elsewhere, in three places: the critic's input (global images tied to the training anatomy), its target (a mixed scalar in which progress can offset danger), and its data (only on-policy experience). The critic-filter baseline and the one-head ablation separate those factors directly.

Why not just train the policy with domain randomization over anatomies? It may work well within the randomized range. Our held-out split, however, extends beyond that range in branch angle, which is where clinical reports locate failures. CathSim's own results show sensitivity to arch type (CathSim). Domain-randomized PPO is run with a matched step budget. Combining it with our filter is also reported, since the filter is complementary rather than a replacement.

Why not just detect collisions from fluoroscopy? The best reported detectors reach only 16.91 mean average precision on vessel-wall collision detection in CathAction, with drops of nearly 30% from phantoms to live animals. Detecting contact after the fact also does not anticipate herniation. We treat vision as a later input, not a substitute.

Proposed Experiments

The first experiment tests anticipation quality. On held-out geometries, we score how well each method's risk signal predicts a failure within $k$ steps, using area under the ROC curve (how well the risk score ranks steps that precede a failure above steps that do not) and median warning lead time, for local-frame versus global-frame anticipators. The decisive check is whether the local-frame model's discrimination degrades much less than the global-frame model's as the test branch angle moves beyond the training range.

The second experiment is closed-loop filtering. We deploy the same PPO policy on held-out geometries under no filter, the force/slack threshold, the critic filter, and our anticipator, alongside domain-randomized PPO with and without our filter. We report cannulation success, herniation and force-violation events per episode, and peak contact force, across seeds.

The third experiment is an attribution check against clinical knowledge. On the aberrant-origin variant, we compute per-step failure impact along scripted trajectories. The model should assign positive failure impact to straight pushing at the origin and negative impact to the twist-and-slow-retract sequence described in the case report (Successful right transbrachial cerebral angiography). If it does not, the model has not learned the mechanics that matter, whatever its aggregate numbers.

A final exploratory check asks whether an image-only version of the failure head, trained on simulated grayscale views, carries any signal on CathAction's phantom collision frames. This would be scored at the frame level, since no comparable frame-level anticipation result was found on Lacuna. The claim fails if, on held-out geometries, the force/slack threshold or domain-randomized PPO matches our filter on failure rate and lead time.