Blur as a Depth Cue for Thin Structures in First-Person Video

Depth networks routinely erase wires, railings, cane tips and tool handles. These objects cover only a few pixels, so a network loses little by painting the background depth over them. The problem is worse for cameras worn on the head or chest. These cameras sit close to such objects, and the wearer's walking and head bobbing smear near-field objects across the image during each exposure. We propose to stop treating that smear as noise. When a camera translates during an exposure, a nearby object blurs more than a distant one, and the blur width is roughly inversely proportional to depth. A thin structure is the best case for measuring this, because its true width is close to zero: almost all of its observed width is blur. Given the camera's own motion during the exposure, which wearable devices log with an inertial sensor, the width of a smeared wire gives its distance.

The concrete proposal is "blur hints." A small module measures the smear profile across detected thin structures and converts it to depth using the logged camera translation, then reports how confident that conversion is. These per-pixel depth estimates selectively supervise a monocular depth network, only where the blur cue is confident and only on thin structures. The hope is that this restores the near-field thin geometry that foundation depth models lose, without disturbing their large-surface accuracy.

Core Claim

In egocentric video, depth-dependent motion blur supplies a single-frame, metric depth signal. It is strongest on thin near-field structures, which is exactly where learned monocular depth is weakest. Distilling it as selective supervision should improve thin-structure depth more than scaling the backbone, adding stereo hints, or deblurring the input first. The physics is simple. Over an exposure of length $T$, a point at depth $Z$ moves in the image by approximately

$$\Delta u \approx \frac{f\, T\, v_\perp}{Z} + \Delta u_{\text{rot}},$$

where $f$ is focal length in pixels, $v_\perp$ is the camera's translational velocity perpendicular to the structure's image direction, and $\Delta u_{\text{rot}}$ is the depth-independent displacement from camera rotation. Gyroscope readings let us remove $\Delta u_{\text{rot}}$. The remaining smear width $w$ then gives $Z \approx f T v_\perp / w$.

A back-of-envelope check shows the cue has usable range. Walking at about 1.4 m/s with a 10 ms exposure and $f \approx 500$ px, a wire at 1 m smears about 7 px and one at 3 m about 2 px. The cue therefore covers the zone where wearers reach, step and collide. Vertical head bob blurs horizontal structures such as railings and cables, and lateral sway blurs vertical posts. Egocentric jitter, usually listed as a nuisance, becomes the source of signal.

Relation to Prior Work

Two papers supply the method. Depth-Aware Image Compositing Model for Parallax Camera Motion Blur models parallax blur as a stack of alpha-blended depth layers. It shows that foreground objects blur far more than background ones under camera translation and close-range "macro" motion. Their model runs the physics forward, taking known depth and trajectory and restoring a sharp image. We run it in reverse, inferring depth from blur, and we also use their compositing model to synthesize realistic parallax blur for training. Self-Supervised Monocular Depth Hints shows that an external depth source from classical stereo matching helps a monocular network escape the local minimum in which thin structures vanish. It does this by supervising only where the hint explains the image better than the network does. We keep that selective-supervision logic but swap the hint source: stereo matching is replaced by blur inversion. Their hints were developed on smooth, forward-moving driving footage. Our setting is the erratic motion where stereo-across-time assumptions break.

Several papers motivate the work by documenting the gap. SANPO reports that ZoeDepth's $\delta_{<1.25}$ falls from 0.97 on KITTI to 0.135 on real egocentric walking footage. The Third and Fourth Monocular Depth Estimation Challenges report that even Depth Anything v2, Marigold and ViT-Giant entries still bleed depth across wires and branches, and those results are on static tripod imagery. EgoM2P trains an egocentric multimodal model that predicts video depth and camera pose together. It does not target thin structures, and its tokenization likely smooths fine boundaries. It is a competitor, not a solution.

Two lines of work constrain the design. Virtual Normal and Plane2Depth add surface-normal and planar priors that flatten thin structures, so we apply no such priors on thin-structure pixels. Deconstructing Self-Supervised Monocular Reconstruction finds that edge-aware smoothness degrades true 3D reconstruction. It also finds that backbone upgrades alone account for much historical progress, which forces us to compare against strong backbones at equal capacity.

Finally, SaccadeCam addresses the small pixel footprint of thin objects by allocating more pixels with steerable optics. We address it by extracting more information from the pixels already captured, with no new hardware beyond the inertial sensor wearables already carry. Classical depth-from-motion-blur exists outside this file. We did not find work in the file that applies it to thin structures in egocentric video or uses it as a training signal for a monocular network.

Research Program

The first component is a blur-width estimator. It is a small network that takes a crop centered on a thin-structure pixel, together with the gyroscope-predicted rotational blur direction and magnitude. It outputs the translational smear width perpendicular to the structure, the structure's intrinsic width, and a confidence. Thin structures come from a segmentation of curvilinear objects. The TopoNet codebase is a candidate starting point for thin-structure masks. The confidence head borrows uncertainty-modeling practice from the Mono-Uncertainty codebase.

The estimator is trained on synthetic data. We take SANPO-Synthetic frames with perfect depth and poses, sample egocentric-style trajectories within each exposure window (walking bob, sway, head rotation), and render parallax blur with the layered compositing model. We add VirtualCMB, which already pairs blur with ground-truth depth.

The second component converts width to depth and handles failure cases. Combining smear width with integrated translational velocity over the exposure gives metric depth. The confidence must collapse in several cases: when camera motion runs parallel to the structure, when rotation dominates, when the exposure is too short for measurable blur, and under rolling-shutter skew. Rolling-shutter skew is modeled from per-row timing where the device exposes it.

The third component distills these depths into a monocular network. Following the depth-hints recipe, a log-L1 loss on thin-structure pixels is applied only where the blur hint has high confidence and explains the observed blurry frame better than the network's current prediction. The explanation test re-renders a local blur patch with the compositing model. Elsewhere the base model is fine-tuned with its usual loss on the same egocentric data. The output is still a single-image depth network, and it needs no inertial input at test time.

Baselines

The naive reference point is Depth Anything v2 run zero-shot on egocentric frames. It shows how much of the problem off-the-shelf foundation models already solve. The closest reported egocentric number is ZoeDepth's $\delta_{<1.25}$ of 0.135 on SANPO-Real from SANPO. For thin structures specifically, no reported result was found on Lacuna for any thin-structure-masked metric on SANPO or Aria Digital Twin. We will establish these numbers ourselves. On the static thin-structure-rich SYNS-Patches, the strongest reported result is a 3D F-Score of 23.72% by PICO-MR in the Third Monocular Depth Estimation Challenge. The ViT-G entry from HRI reached 23.05% in the Fourth. We report SYNS-Patches to confirm the method does not hurt static performance.

The strongest competitor is the same ViT-Large Depth Anything v2 model fine-tuned on identical egocentric training data (SANPO-Real, SANPO-Synthetic, Aria Digital Twin training sequences). It receives the same compute, steps and learning-rate sweep, and is supervised by available ground truth or stereo depth. This is what a practitioner would do, and it isolates whether our signal adds anything beyond domain adaptation.

The second competitor is the depth-hints method of Watson et al., with semi-global matching hints computed on SANPO's real stereo pairs. It is the best existing thin-structure supervision source and shares our selective-loss machinery, so the comparison isolates the hint source. EgoM2P is evaluated as released on Aria Digital Twin, which it already uses, as the strongest egocentric-specific depth model.

Two ablations isolate the key ingredient. The first is blur augmentation alone: training on blur-augmented frames with no hints tests whether the network learns the cue implicitly. The second is hints computed from a shuffled inertial trajectory taken from a different clip: if the gains survive wrong motion data, the improvement is not coming from the physics.

Beating the competitors means a statistically significant reduction in AbsRel on thin-structure pixels within 3 m on held-out Aria Digital Twin sequences and synthetic-blur SANPO. We pre-register a margin of at least 15% relative to the fine-tuned foundation model. Full-image AbsRel must stay within 2% of that model.

Why not just…?

Why not just use a bigger, egocentric-fine-tuned foundation model? Backbone scale is the most reliable source of progress per Spencer et al.. Even so, the challenge reports find thin-structure bleeding in every ViT-G-class entry on static imagery. The underlying problem is that losses barely penalize erasing a few pixels, and that does not change with capacity. This is a real threat on near-field thin structures in egocentric data, which no paper in the file measures. The equal-budget fine-tuned baseline settles it directly. If that baseline matches us, the idea fails.

Why not just use stereo or multi-frame matching hints? SANPO and Aria devices have stereo. Matching volumes are downsampled, however, and lose structures thinner than their stride. Near-field parallax blur also smears the very edges that matching depends on. Across time, erratic head rotation breaks narrow-baseline assumptions. Stereo is still strong where blur is weak, so we treat it as a competitor and test it head to head. We also report a union of stereo and blur hints, since the two signals may simply be complementary, and that would be a legitimate finding.

Why not just deblur first, then estimate depth? Torres and Kämäräinen show that standard deblurring networks such as Restormer and MPRNet produce halos at depth discontinuities under parallax, which is exactly where thin structures sit. Their depth-aware model avoids this, but only by requiring depth as input, which is circular for us. We include Restormer followed by the fine-tuned model as a baseline and expect it to degrade thin-structure boundaries.

Why not just train on blur-augmented data and let the network learn the cue? A single blurry frame cannot separate a near object under slow motion from a far object under fast motion. Without the camera's velocity, the blur-depth relation is not identifiable. This is why the inertial sensor is needed at training time. The blur-augmentation ablation tests this, as does a variant that feeds inertial velocity as an input channel. If the motion-conditioned network matches the hint approach, we would adopt it as a simpler design, at the cost of requiring the inertial sensor at test time.

Proposed Experiments and Decisive Checks

The first check comes before any training. On Aria Digital Twin and RealCMB, which have measured depth, poses and real blur, we measure the fraction of thin-structure pixels within 3 m whose smear exceeds 1.5 px. If daylight exposures on SANPO-Real are too short, the cue will be largely absent outdoors. In that case we restrict the claim to indoor and low-light wearable use rather than overstate it.

The second check validates the estimator. On RealCMB and held-out synthetic data, we plot blur-hint depth against measured depth as a function of confidence. We expect the error to grow roughly with $Z^2$, because $w \propto 1/Z$ means a fixed width error produces a depth error proportional to $Z^2/(f T v_\perp)$. Seeing that growth would confirm the physics is doing the work.

The main comparison is thin-structure-masked AbsRel and $\delta_{<1.25}$, broken down by depth bin and by the angle between motion and structure. Gains should concentrate where motion is perpendicular to the structure and vanish where it is parallel. That angular signature is the clearest evidence that the blur cue, not extra data, drives the improvement. The shuffled-motion ablation provides the complementary negative control.

Finally, we report 3D F-Score on SYNS-Patches to confirm no static regression. As a downstream check, we feed the improved depth into EgoTrack3D on Aria Digital Twin and test whether thin handheld objects are tracked more accurately in 3D.