Many robot learning policies now output a short plan of several future actions at once, called an action chunk, instead of one action per control step. This makes motion smoother and stops small errors from compounding. It also leaves the robot partly blind until the next plan arrives. Underwater, that blindness fails in two different ways that call for opposite fixes. When a current surge pushes the vehicle off course mid-chunk, the robot should drop the rest of the plan and replan at once. When turbidity, backscatter, or a submerged structure blinds the camera or sonar, replanning from those degraded images is the wrong move, and the robot should keep following its last good plan. Current methods for deciding when to cut a chunk short use a single uncertainty score, and that score rises in both cases. We think this is the main reason chunked policies will not carry over naively to underwater vehicles.
We propose a chunked policy for an autonomous underwater vehicle (AUV) whose replanning is controlled by two separate signals. The first is a dynamics-surprise signal: the gap between the motion the vehicle commanded and the motion its velocity and inertial sensors report. These sensors keep working when vision is occluded. The second is a perception-reliability signal from the visual and sonar encoder. Dynamics surprise decides when to truncate a chunk. Perception reliability decides what a replan may condition on: fresh observations when they are trustworthy, and the remaining tail of the previous chunk when they are not. The whole system is trained in simulation with randomized hydrodynamics and randomized visual degradation, so that it can be deployed on a vehicle with no real-world training.
Core Claim
Uncertainty-triggered chunk truncation, as done with entropy across sampled trajectories in Adaptive Action Chunking Strategy from World Feedback in Mixed Traffic or with cross-horizon consensus in Mixture of Horizons in Action Chunking, mixes up two causes of uncertainty. One is that the world has changed: an unmodeled force has moved the vehicle. The other is that the robot cannot see. Under combined currents and occlusion, a single gate either replans too often from corrupted observations or replans too rarely when it is being pushed off course. Separating the two causes, and taking the "world changed" signal from proprioception rather than vision, should give higher success and fewer collisions than single-signal gating. The gain should come mainly from the occluded conditions, which is a falsifiable prediction.
What Prior Work Gives Us and Where It Stops
The case for chunking comes from imitation learning on manipulators. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware and Diffusion Policy predict multi-step action blocks to avoid compounding errors and averaging between conflicting demonstrations. Action Chunking and Data Augmentation Yield Exponential Improvements in Behavior Cloning for Continuous Spaces shows that chunking avoids exponential error growth. These papers motivate the policy class. None of them faces an external force large enough to make a plan obsolete within its own horizon.
Three papers supply the execution machinery. UA2C samples 32 flow-matching chunks and executes only the prefix on which the samples agree. Mixture of Horizons fuses chunks of 10, 20, and 30 steps and executes based on agreement across horizons. Its tests include the heavy-occlusion settings of RoboTwin 2.0, but only with on-board cameras on land and with nothing that displaces the robot. Real-Time Robot Execution with Masked Action Chunking (REMAC) trains with masked prefixes so that a new chunk stays consistent with actions already committed during inference delay. We reuse its prefix-preserving idea as the "trust the old plan" branch of our replanning rule.
The underwater work supplies the disturbance signal and the baselines, but not chunking. Current-Inference Transformer Policies for Robust AUV Navigation from World Feedback infers hidden ocean currents from the mismatch between commanded actions and observed outcomes over a history window. It supervises that inference with the simulator's true current vectors during training, and adds a per-step reinforcement-learning correction on top of Line-of-Sight guidance. It does not chunk, does not model sensing dropouts, and is evaluated only in simulation. Make Your AUV Adaptive and USV-AUV Collaboration Framework feed flow or current state into per-step SAC, TD3, and DDPG controllers and assume that state is directly available.
Our closest prior, then, is Current-Inference's mismatch signal. We repurpose it from a per-step correction into a replanning trigger, and pair it with a perception gate that the underwater work never needed because it assumed clean sensing.
Two lines of work constrain the design. First, collaborative perception that sees around occluders by sharing neural features between agents, such as Where2comm and CoBEVT, assumes radio links carrying megabits per second. Underwater acoustic modems often carry under 10 kbps with long propagation delays, so we deliberately treat occlusion as a single-vehicle problem. Second, Asymmetric Actor Critic for Image-Based Robot Learning shows that giving privileged state only to the training-time learner, while the deployed policy sees realistic sensors, enables zero-shot transfer. We adopt the same asymmetry. Visual randomization follows CAD$^2$RL and dynamics randomization follows Sim-to-Real: Learning Agile Locomotion For Quadruped Robots.
Research Program
Action space. A chunk is a sequence of $k$ world-frame waypoints with velocity targets, not raw thruster commands. An inner-loop tracking controller with an extended state observer turns waypoints into thrust at high rate, in the style of the super-twisting observer controllers in Three-Dimensional Trajectory Tracking Control of Underactuated AUV. This choice is deliberate. Spatial-trajectory actions transfer better across the reality gap (Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space). It also absorbs the obvious reviewer objection that a feedback inner loop already rejects small disturbances. What the inner loop cannot fix is a plan that has become wrong: a waypoint sequence laid out for a current that has since reversed, or a route that now passes too close to a structure the vehicle has drifted toward. Deciding when the plan is wrong is the job of the gate.
Gate. Let $\hat v_t$ be the vehicle velocity predicted from commanded thrust and nominal dynamics, and $v_t$ the velocity measured by the Doppler velocity log and inertial unit. A causal transformer over the last $K$ steps, trained as in Current-Inference with an auxiliary head supervised by the true simulated current, produces a disturbance-surprise score $d_t$. This score is the change in the inferred current since the current chunk was planned, normalized by the inferred current's uncertainty. A reliability head on the sonar and camera encoder produces $c_t \in [0,1]$. It is trained in simulation against the fraction of task-relevant geometry that is actually visible, which is known exactly from the renderer. Execution then follows
$$\text{replan at step } t \iff d_t > \tau_d, \qquad \text{condition replan on } \begin{cases} o_t & c_t \ge \tau_c \\ (o_{\text{last good}},\ \text{remaining chunk tail},\ \text{current history}) & c_t < \tau_c \end{cases}$$
Here $o_t$ is the current sensor observation and $\tau_d, \tau_c$ are thresholds tuned in simulation. The low-reliability branch re-anchors the old plan to the vehicle's new measured pose and current estimate. It uses REMAC-style prefix-preserved sampling, so the replan corrects for drift without trusting what it currently sees. A learned variant, in which the policy outputs its own execution length conditioned on $(d_t, c_t)$, is a secondary arm.
Training. The teacher is a privileged planner in simulation that sees the true flow field and true obstacle geometry. The student is a flow-matching chunk policy, meaning a generator trained to transform noise into action sequences. It sees only sonar, camera, velocity-log, and inertial histories. Training is behavior cloning with on-policy relabeling, plus the noise-injection augmentation that the chunking theory paper identifies as necessary.
Simulation. Vehicle dynamics follow the REMUS-style 6-DOF model described for Current-Inference: added mass, Coriolis terms, cross-flow drag, thruster saturation, and actuator lag. Its added mass, drag coefficients, thruster lag, and current statistics are randomized per episode. Currents include the tidal shallow-water and Lamb-vortex models used in the USV-AUV work. Perception is rendered in OceanSim, whose physics-based attenuation, backscatter, and real-time imaging sonar (28–31 FPS reported) make per-episode turbidity randomization affordable. OceanSim has no hydrodynamics engine, so we couple it to the dynamics model, which is an engineering contribution in its own right. Occlusion comes from randomized pilings, wreck geometry, and particulate plumes.
Baselines
The naive reference is Line-of-Sight guidance alone. In the Current-Inference paper's 1.5x intensified-current stress test it reached a 37% success rate with a 56% timeout rate, which shows how much headroom learning has.
The strongest existing underwater method is the Current-Inference Transformer itself: per-step residual reinforcement learning on top of guidance, which reached 65% success and 29% timeouts in that same test, with slightly more collisions than guidance alone. We reimplement it in our coupled simulator. It gets the same randomized dynamics and the same history length, but perfect obstacle coordinates replaced by our degraded perception. Its reported 65% is therefore a reference point from a different simulator, not a number we can compare against directly. Beating it means higher success at equal or lower collision rate on the same randomized test episodes. Every learned method receives the same training environment steps and the same hyperparameter search budget, and is evaluated on identical held-out episode seeds.
The strongest existing chunk-execution baseline is UA2C-style entropy truncation applied to our exact policy. It uses 32 samples and executes the agreed prefix, with the same action space and inner loop, so only the gating rule differs. We also run a Mixture-of-Horizons consensus variant. In a direct comparison, UA2C reported a 13.17% reward gain over fixed-length chunking in traffic, so this is a serious competitor.
Three ablations isolate the key ingredient. The first is a fixed-length chunk with periodic replanning and the same waypoint inner loop with observer, which tests whether any gating is needed. The second keeps only the disturbance gate and always replans from the fresh observation. The third keeps only the reliability gate. The proposal wins only if the two-signal gate beats entropy truncation in the conditions that combine currents with occlusion, and if removing the reliability branch erases most of that gain.
Why not just…?
Why not just keep the per-step residual policy from Current-Inference, which already handles currents? It does handle currents, but it re-decides at every step from whatever it observes. Under occlusion, every step's decision inherits the corrupted observation. It also brings back the per-step jitter that chunking removes, and its reported collision rate already rose above guidance-only. Our factorial evaluation checks this directly. If the per-step policy matches ours under occlusion, the chunking part of the proposal is unnecessary and we will say so.
Why not just use waypoint chunks with a strong disturbance-observer inner loop and replan on a fixed clock? This is the most serious threat, so it is built into the method's action space and kept as an ablation. It may well match us in currents alone, because the inner loop corrects small drift. We expect it to fail in two places. When drift makes the plan itself invalid between clock ticks, it acts too late. When a fixed-clock replan lands during a blind period, it replans from bad observations. If it matches the two-signal gate across the combined grid, the gate adds nothing, and that outcome would be the paper's finding.
Why not just use UA2C's entropy-based truncation, which is already adaptive? Entropy across sampled plans rises when observations are degraded, not only when the world has changed. So under turbidity it should truncate more often and replan from poor inputs, which is exactly the wrong response. The decisive test is the occlusion-only condition. If entropy truncation degrades less than we predict there, the conflation argument is wrong.
Why not just randomize occlusion heavily during training so the policy learns robustness? We do randomize it. But randomization makes the encoder tolerant to degradation without telling the executor when replanning is harmful. The reliability-gate-only ablation, compared against a heavily randomized policy without gates, separates these two effects.
Why not tube-based MPC with barrier functions? These methods carry formal guarantees, and Wei Wang et al. report a 40% reduction in river tracking error. They do, however, need an explicit dynamics model and obstacle geometry. Under occlusion the obstacle geometry is exactly what is missing. We include a tube-MPC planner with access only to perceived obstacles as an additional reference, where compute allows.
Proposed Experiments
The main experiment is a factorial grid in held-out simulation. One axis is current intensity: nominal, 1.5x, and surge events. The other is perceptual degradation: clear, turbid, and structural occlusion. Every method in the baselines section runs on 100 randomized episodes per cell, mirroring the Current-Inference protocol. Metrics are success, collision, and timeout rates, plus energy (integrated thrust), cumulative jerk, and replans per minute.
The decisive check is the interaction between the two axes. In the current-only cells, the two-signal gate and entropy truncation should perform similarly. In the occlusion and combined cells, the two-signal gate should lead, and replans issued during low-reliability periods should account for most of entropy truncation's collisions. If the gap does not appear, or appears equally in the current-only cells, the core claim fails.
A second check probes the gate's signals. We correlate $d_t$ with true current changes, and $c_t$ with true visible fraction, on held-out randomized dynamics, including hydrodynamic parameters outside the training range.
Finally, as a zero-shot transfer probe, we deploy the policy without fine-tuning in a towing tank with generated flow and artificial turbidity. A digital twin of the Michigan MHL tank already exists in OceanSim. We measure station keeping and short inspection passes against the guidance and inner-loop baselines. No reported result was found on Lacuna for any learned AUV policy transferred zero-shot to hardware, so this is exploratory and we will report raw outcomes.
References
- Adaptive Action Chunking Strategy from World Feedback in Mixed Traffic
- Mixture of Horizons in Action Chunking
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- Action Chunking and Data Augmentation Yield Exponential Improvements in Behavior Cloning for Continuous Spaces
- Real-Time Robot Execution with Masked Action Chunking
- Current-Inference Transformer Policies for Robust AUV Navigation from World Feedback
- Make Your AUV Adaptive: An Environment-Aware Reinforcement Learning Framework For Underwater Tasks
- USV-AUV Collaboration Framework for Underwater Tasks under Extreme Sea Conditions
- Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
- CoBEVT: Cooperative Bird’s Eye View Semantic Segmentation with Sparse Transformers
- Asymmetric Actor Critic for Image-Based Robot Learning
- CAD2RL: Real Single-Image Flight Without a Single Real Image
- Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
- Three-Dimensional Trajectory Tracking Control of Underactuated AUV Based on Fractional-Order PID and Super-Twisting Extended State Observer
- Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks
- OceanSim: A GPU-Accelerated Underwater Robot Perception Simulation Framework
- Robust Model Predictive Control with Control Barrier Functions for Autonomous Surface Vessels