Moving the Elbow Out of the Way: Uncertainty-Directed Null-Space Un-Occlusion for Grasping

A seven-joint robot arm has one more joint than it needs to place its hand at a given position and orientation. That spare joint lets the elbow and forearm swing through a continuous family of postures while the hand stays on the same path. Meanwhile, the most common way a fixed-base arm loses sight of what it is grabbing is that its own forearm and wrist pass between the external camera and the object during the final approach. We propose to use the spare degree of freedom to keep the camera's view clear during that approach. The arm should not clear the view of the whole object, though. It should clear only the specific pixels that would settle the robot's remaining doubt about how to grasp.

In practice, the robot keeps a set of sampled guesses about the object's 6D pose, meaning its position and orientation in 3D. It renders what the camera would see under each guess and marks the pixels where the guesses disagree in ways that would change the chosen grasp. It then pushes its elbow, within the spare posture freedom, so that the arm stops covering those pixels. When the guesses agree, the arm leaves its posture alone. The claim to test is whether steering by uncertainty in this way beats the obvious alternative of steering by plain geometric visibility, and whether it can match adding more cameras.

Core Claim

Occlusion-avoidance cost should be weighted by how informative each pixel is about the decision the robot has to make, not by whether the pixel belongs to the object. The two differ most for symmetric or partially hidden objects. A mug body is highly visible and nearly useless for locating the handle. A small patch where the handle might be is barely visible and decides the grasp.

We control the arm with the standard redundancy-resolution law:

$$\dot q = J^{\dagger}\dot x_{\text{task}} + (I - J^{\dagger}J)\,(-k\,\nabla_q C(q)).$$

Here $q$ is the joint vector. $J$ is the end-effector Jacobian, the matrix mapping joint velocities to hand velocities, and $J^{\dagger}$ is its pseudo-inverse. $\dot x_{\text{task}}$ is the commanded hand velocity from any upstream grasp planner, and $k$ is a gain. The projector $(I - J^{\dagger}J)$ ensures the second term moves only the elbow family and never the hand. The cost is

$$C(q) = \sum_{p} w_p\, o_p(q),$$

where $o_p(q)\in[0,1]$ is a soft, differentiable measure of how much the arm's link capsules, projected into the camera image, cover pixel $p$ at configuration $q$. The weight $w_p$ is the disagreement at pixel $p$ among rendered depth images of pose hypotheses that lead to different grasps. It is zero wherever all grasp-relevant hypotheses predict the same appearance. As the pose distribution collapses, all $w_p$ go to zero and the arm returns to its nominal posture.

Relation to Prior Work

The closest work falls into two groups: one that ties uncertainty to occlusion, and one that uses redundancy or visibility without uncertainty.

In the first group, Uncertainty-Aware Non-Prehensile Manipulation with Mobile Manipulators under Object-Induced Occlusion trains an RL policy whose collision-uncertainty signal drives a mobile base to "peek" sideways around a pushed box. It uses 2D LiDAR in simulation only. We keep its central insight that variance should drive perception-seeking motion. We move that insight to arm self-occlusion during prehensile grasping, with RGB-D input, and we use an analytic controller instead of a learned policy. SE(3)-PoseFlow supplies our pose-hypothesis generator, a flow-matching model that samples 6D poses. It already uses uncertainty downstream through grasp averaging and by moving the wrist camera to reduce rotational covariance. Moving the wrist camera changes the hand path. We leave the hand path fixed and reduce uncertainty for free through posture alone, which is what makes our method compatible with any upstream grasp planner.

In the second group, Redundancy Resolution as Action Bias supplies the null-space mechanism, projecting secondary objectives into the redundant joints without hurting the primary task. It only ever uses posture or safety objectives. MobileH2R penalizes arm configurations that block the camera's view of the target. It does so offline, when generating demonstrations, with a deterministic visibility loss. Its approach is the natural geometric version of our idea and serves as our main ablation. Active Vision Might Be All You Need motivates the problem from the hardware side: a dedicated camera arm beats static cameras on occlusion-heavy tasks, but it adds cost, and simply stacking more cameras can degrade learned policies. ActiveVLA and RVT render virtual views from point clouds, which cannot recover points the sensor never saw. That limitation is exactly the self-occlusion case we target. ContactGuard detects imminent pre-contact failure but can only abort. Our controller is a constructive counterpart that acts before the blind spot forms, though it does nothing about finger occlusion once the hand is at the object.

Research Program

We would build three components.

The first is a pose-hypothesis module. It retrains SE(3)-PoseFlow on objects rendered in RLBench and produces roughly 32 samples per frame from external RGB-D. Each hypothesis is labeled with the grasp the upstream planner would choose for it.

The second is a disagreement renderer. It rasterizes each hypothesis's object mesh from every external camera and computes $w_p$ as the depth variance across hypotheses, restricted to pairs whose chosen grasps differ by more than a threshold.

The third is the null-space controller. It computes $\nabla_q C$ through differentiable projection of link capsules, so no neural network sits in the control loop. Weights are refreshed at the pose model's rate, and the controller runs at arm rate.

The analysis question is when this beats plain geometric visibility. Our hypothesis is that the gap grows with object symmetry and with the fraction of the object's projected area that carries no information about the grasp.

We will also measure the method's limits. We will record how often the null space simply cannot clear the key pixels given the camera's position relative to the arm. That rate bounds the method, and it decides where an extra camera is genuinely necessary.

Baselines

The naive reference fixes the elbow at its nominal redundancy resolution and uses the same PoseFlow pose, frozen at the last confident estimate. This is the open-loop blind execution a reviewer would expect to fail under occlusion. On the real mug task, SE(3)-PoseFlow reports 95% grasp success versus 75% for a deterministic-pose baseline. We will reproduce both numbers as the starting reference.

The strongest existing uncertainty-aware method is PoseFlow's own covariance-minimizing active perception, which moves the eye-in-hand camera. We run it with the same pose model, the same grasp planner, and the same wall-clock budget per episode. Beating it means matching its success rate while adding less approach time, or exceeding its success rate in scenes where the wrist camera sees nothing useful.

The key ablation is geometric visibility. It uses the same null-space controller with $w_p$ set to one on every pixel the object's mean pose projects to, which is a runtime version of MobileH2R's visibility loss. Everything else is identical, so this ablation isolates the uncertainty weighting. The proposal survives only if uncertainty weighting wins on the ambiguous-object subset.

We also compare against a hardware alternative: our base setup plus one additional static camera, fused into the same pose model. For learned-policy context on RLBench, ActiveVLA reports 91.8% average success, versus 81.4% for RVT-2. It also reports 65.9% on COLOSSEUM. These numbers come from full standard task suites, not from our occlusion subset. We will re-run ActiveVLA on our subset rather than compare against its reported figure.

For the dedicated camera arm, there is no reported numerical result on Lacuna for AV-ALOHA's occlusion tasks. We treat it qualitatively and do not claim to beat it.

Why not just…?

Why not just add a second static camera? This is the most serious threat. For many layouts, one more camera may remove self-occlusion outright. Evidence from Active Vision Might Be All You Need that extra cameras can hurt applies to learned policies. It applies less to our pose-estimation pipeline, which fuses views explicitly. We therefore run the extra-camera baseline directly. We also report the fraction of placements for the second camera under which our single-camera method matches it. If a randomly placed second camera routinely matches us, the contribution shrinks to a cost argument, and we will say so.

Why not just maximize visibility of the object? Geometric visibility treats all object pixels alike. For symmetric objects it will happily clear the view of an uninformative body while leaving the handle region covered. The geometric-visibility ablation tests this directly. If it ties with our method even on symmetric objects, the uncertainty weighting is unnecessary, and the paper's claim fails.

Why not just stop and look with the wrist camera before approaching? Pausing works, but it costs time and changes the hand path. It is also roughly what PoseFlow's active perception does. We compare success rate and total approach time against it. Our advantage exists only if the null-space motion runs during the approach, not before it.

Proposed Experiments

We will build an occlusion subset of RLBench (RLBench) using the Franka arm and a single shoulder RGB-D camera. The subset includes tasks where the approach path passes the forearm through the camera's line of sight, such as picking a mug by its handle, grasping a lid, or inserting a peg from behind. Object sets are stratified into symmetric-ambiguous and asymmetric groups.

We report grasp and task success, time to success, and the fraction of approach frames on which the posterior's grasp-relevant spread falls below threshold. We also check the pose module on LINEMOD-Occlusion against PoseFlow's reported 35.2% accuracy at the 5°/5 cm threshold, to confirm that our retraining did not degrade it.

To test robustness, we add COLOSSEUM's camera-pose and distractor perturbations (THE COLOSSEUM). These matter because our null-space cost depends on knowing the camera extrinsics, the camera's position and orientation relative to the robot.

On a real Franka arm, we replicate the occluded-mug-handle grasp from PoseFlow, then add a lid and a peg task.

The decisive check is the interaction between method and object group. We predict that uncertainty weighting ties with geometric visibility on asymmetric objects and beats it on symmetric-ambiguous ones. If the gap does not appear on the ambiguous group, the idea reduces to runtime visibility control, and we would report it as that.