Tissue Strain Dose from Surgical Video as a Postoperative Delirium Exposure

Postoperative delirium is an acute state of confusion and inattention that hits many older patients after surgery. It predicts longer hospital stays, later dementia and death. The leading explanation is a "two hit" collision. Some patients arrive with low reserve (brain atrophy, frailty, poor cognition), and surgery then delivers a physical and inflammatory stress. Prediction models measure the first hit fairly well, but they measure the second hit only through distant stand-ins such as anesthesia time, blood-pressure swings and opioid doses. None of them measure what the surgeon physically does to tissue: how much is pulled, stretched and compressed, for how long, and how hard.

In minimally invasive and robotic surgery, that quantity is sitting in the endoscope video. Computer-vision methods can already rebuild deforming tissue in 3D over time from that video, but so far only to draw overlays and guide instruments. We propose to turn those reconstructions into a per-patient mechanical dose: accumulated tissue strain, meaning how much tissue is stretched relative to its resting length, integrated over the operation. We then test whether this dose predicts delirium beyond everything current models already use, and whether its effect is larger in low-reserve patients.

A useful property makes this feasible from a single camera. Strain is a ratio of lengths, so the unknown global scale of monocular depth estimates cancels out of it. That scale ambiguity is the usual obstacle to getting physical measurements from monocular endoscopy.

Core Claim

Our claim has three parts. First, a scale-invariant strain dose computed from 4D reconstruction of endoscopic video carries information about delirium risk that is not captured by surgery duration, procedure type, blood loss, or the fine-grained intraoperative vital signs modeled by the best current system. Second, this dose interacts with baseline reserve, which is the two-hit hypothesis stated as a testable interaction. Third, part of the early blood-pressure instability that current models rely on is a downstream readout of tissue handling rather than an independent signal.

The idea is new because no delirium model uses operative-site mechanics at all. It also rejects the obvious engineering route of feeding raw video into a multimodal network. That route is the setting where auxiliary modalities are known to hurt: in Autoregressive EHR Foundation Models with Multimodal Inputs, adding chest radiographs lowered mortality AUROC from 0.8651 to 0.8624, below the code-only model. Instead, we compress video into a few physically meaningful time series before fusion.

We also treat the choice of estimator as an open question, not a premise. Cheaper proxies may carry most of the signal, and the program is built so that we find out.

Research Program

Reconstruction. The method layer reuses existing deformable reconstruction work. Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane (Forplane) splits the 4D volume into three static spatial planes and three space-time planes. It reaches PSNR 37.31 on EndoNeRF prostatectomy clips and trains in about 10 minutes per clip. Endo-4DGS represents tissue as moving 3D Gaussians, uses Depth-Anything as a relative-depth prior, and renders at 100 FPS. FLex jointly estimates camera motion and deformation over local time windows.

These papers supply the machinery, but they were built for rendering fidelity on short clips. Whole operations last hours, so we run overlapping windowed reconstructions in the style of FLex, entirely offline after surgery.

Strain dose. From each window's deformation field we compute the local Green–Lagrange strain tensor on the reconstructed surface. This tensor measures how much a small patch of tissue has stretched or sheared relative to its reference state. Writing $E(x,t)$ for that tensor at surface point $x$ and time $t$, we define the dose as

$$D = \int_0^T \int_{S} \mathbf{1}\left[\lVert E(x,t)\rVert > \tau\right] \, \lVert E(x,t)\rVert \, dA \, dt,$$

where $S$ is the visible tissue surface, $T$ is operative time, and $\tau$ is a threshold. The threshold removes the small, cyclic strain caused by breathing and heartbeat, which we identify by its periodicity. We also keep the dose as a minute-resolution time series $D(t)$ and record peak strain rate.

Because $E$ is built from ratios of lengths, a global depth rescaling leaves it unchanged. This is why the unscaled depth priors used by Endo-4DGS are acceptable here. Topology changes from cutting and optical degradation from smoke, blood and glare remain real threats; EndoMD-SLAM documents how photometric methods fail under these conditions. We therefore mask frames with low rendering confidence and report the fraction of each operation that could be measured.

Validating the measurement. Before any clinical analysis, we check that $D$ measures what it claims. We use the Hamlyn phantom sequences and bench recordings of silicone phantoms stretched by known amounts. On robotic cases we also compare $D$ against instrument grasp and retraction events from the robot's kinematic logs. We will not report a clinical association with a quantity whose physical meaning has not been checked.

Cohort. The clinical layer is a prospective multi-center cohort of older adults undergoing robotic or laparoscopic gastrointestinal cancer resection. This is the population in which Bo Dong et al. found ultrasound-defined sarcopenia associated with roughly four-fold delirium risk. Reserve is measured by MMSE, education, sleep quality and pain scores as in Xu Lin et al., plus bedside muscle ultrasound. Where a routine preoperative head CT exists, we add ventricular enlargement following Iván Murrieta-Álvarez et al..

Intraoperative vitals and drug infusions are recorded at full resolution. Delirium is assessed with the Confusion Assessment Method twice daily through postoperative day 7. This design responds to a known weakness of the TRAPOD work, Applying a transformer architecture to intraoperative temporal dynamics improves the prediction of postoperative delirium, which scored delirium only once, in the recovery room, with the Nursing Delirium Screening Scale and so could miss delayed and hypoactive cases. Our target is about 1,200 patients across at least three centers. At the 15–20% incidence reported in the cardiac and non-cardiac cohorts this should yield roughly 200 events, enough for the interaction analysis. Because the single-center limitation applies to every existing model, the analysis uses leave-one-center-out validation.

Fusion. Rather than cross-attending to video tokens, $D(t)$, strain rate and measurable-fraction enter a TRAPOD-style encoder as three extra channels aligned to the vitals clock. The reserve variables enter as static tokens. In a separate logistic model we fit an explicit dose × reserve interaction term for the scientific test.

Mechanism. Finally, we test the mediation question raised by Xiao Shen et al.. They found that postoperative systolic variability raised risk (OR 1.17) while intraoperative variability appeared protective (OR 0.92), and they attributed the reversal to smoothing during cardiopulmonary bypass. We align strain spikes with arterial pressure and ask whether handling events are followed within minutes by systolic changes. We then ask how much of the early-surgery blood-pressure signal that TRAPOD relies on disappears once dose is included.

Baselines

The naive reference point is a model with only procedure type and anesthesia duration. Both are what current risk tools use to stand in for surgical stress, and the dose must beat this trivially cheap summary.

Above it sits a refit of the static nomogram of Xu Lin et al.. That model combines age, MMSE, diabetes, education, sleep and pain scores, ASA class and anesthesia time, and it reported an internal validation AUC of 0.939 with 94.44% sensitivity. That figure comes from 199 validation patients at one center, so we expect it to shrink under leave-one-center-out testing. We refit the model's coefficients on our training folds rather than reuse the published formula.

The strongest competitor is TRAPOD, the encoder-only transformer over intraoperative vitals and drug titration. On 72,100 surgeries it reported mean AUROC 0.774 and AUPRC 0.330 from the first 30 minutes of surgery, outperforming LSTM, MLP and logistic regression baselines. It is the strongest because it already captures the dynamic hemodynamic and opioid signals that a dose effect could be confounded with. We reimplement it on our cohort with identical inputs, the same patient splits and the same hyperparameter search budget. Our model is the same architecture plus the dose channels, so any gain can only come from the dose.

The ablations isolate the key ingredient by replacing the 4D strain dose with progressively cheaper estimators under identical fusion:

  • 2D optical-flow magnitude over tissue pixels;
  • robot instrument path length and grasp time from kinematic logs;
  • a generic surgical video embedding pooled per minute from a surgical vision-language model such as SurgLLM.

A separate ablation feeds raw video features through Flamingo-style gated cross-attention. This tests whether our compression step is what avoids modality competition.

Beating TRAPOD means an improvement in AUPRC whose bootstrap 95% interval excludes zero in pooled out-of-center predictions, together with a significant likelihood-ratio test for the dose terms. Beating the cheap proxies means the same criterion applied between the strain model and each proxy model.

Why not just…?

Why not just use duration, blood loss and a nomogram? Longer and bloodier operations involve more handling, so these may already absorb the dose signal. This is a real threat, and the naive and nomogram baselines test it directly. We also run the analysis within each procedure type and adjust for duration and blood loss. If the dose adds nothing after that adjustment, it was only measuring how extensive the surgery was, and we will report that as the finding. The existing evidence that static summaries lose information is TRAPOD's advantage over models built on aggregated features, but that evidence concerns vitals, not handling.

Why not just compute 2D optical flow, or use robot kinematics, instead of 4D reconstruction? Either may work nearly as well, which is why the program treats them as competing estimators rather than straw men. Our reason to expect 4D to help is specific. 2D flow mixes camera motion with tissue motion and changes with viewing distance. Instrument logs record what the tool did, not how much the tissue deformed; grasping fat and retracting bowel produce the same log trace. Strain from a reconstruction that separates camera motion from deformation, as FLex does, is free of both confounds. If the flow or kinematics proxy matches the 4D model within the confidence interval, the practical recommendation becomes the cheaper estimator. The scientific claim, that handling is an exposure, still stands, and we would say so.

Why not just feed the video end to end into a multimodal model? The closest evidence argues against it. Adding a sparse, high-dimensional modality to a dense, predictive one degraded performance in Autoregressive EHR Foundation Models with Multimodal Inputs. PheME needed separate per-modality predictors in an ensemble just to balance notes against structured codes. With roughly 200 events, an end-to-end video model would also overfit. The gated cross-attention ablation tests this directly. If it matches the compressed dose, compression was unnecessary, but it would also give up the interpretable exposure needed for the interaction and mediation analyses.

Proposed Experiments and Decisive Checks

The first experiment validates the measurement. On phantoms with known imposed strain, we report the correlation and bias between measured and applied strain. We report the same for synthetic global depth rescaling, to confirm the claimed scale invariance, and for smoke and glare-corrupted segments. As reconstruction sanity checks we report EndoNeRF PSNR against Forplane's 37.31. No reported result was found on Lacuna for strain accuracy on any of these datasets, so this measurement is itself a contribution.

The second experiment is the primary clinical comparison: leave-one-center-out AUROC and AUPRC for the strain model against TRAPOD and the proxies, with calibration curves.

The third experiment tests the two-hit hypothesis. Its estimand is the dose × reserve interaction coefficient in a model adjusted for procedure type, duration, blood loss, opioid dose and hemodynamic variability.

The fourth experiment tests mechanism. It measures pressure responses locked to handling events and the proportion of TRAPOD's early blood-pressure attribution that is taken over by dose.

Three negative controls must behave as expected:

  • Shuffled videos. A dose computed from another patient's video of the same procedure type must lose all incremental value.
  • Camera motion only. A dose computed only from camera motion must carry no signal.
  • Dose–response shape. The dose–response curve must be monotone rather than driven by a few extreme cases.

The proposal succeeds if the strain dose survives the within-procedure adjustment, beats TRAPOD out of center, and beats at least the optical-flow proxy. It fails informatively if a cheap proxy matches it, and it fails outright if the shuffled-video control performs as well as the real dose.