When a camera or object moves while the shutter is open, each pixel records an average of everything that passed through it during the exposure. Category-level pose estimators do not account for this. These are systems that recover an unseen object's 3D rotation, position and size without a CAD model of that specific object, by predicting for every pixel where that pixel sits on a shared "canonical" version of the object category. They assume each pixel maps to one point on that canonical object. Under blur this assumption is false. A blurred pixel saw a short path of surface points sliding past it, so asking a network for a single canonical coordinate forces it to output an average of that path. That average often lies inside the object rather than on its surface, and the pose fitted to these coordinates degrades.
We propose to predict the path instead. For each pixel, the network outputs where on the canonical object that pixel was looking at the start, middle and end of the exposure, together with how much of the exposure the object actually covered that pixel. A small solver then fits two object poses, one at shutter-open and one at shutter-close, that explain all these per-pixel paths at once. In plain terms, the network reads each blur streak as a record of motion instead of treating it as damage. The output is a pose and also an intra-frame velocity, both recovered from one frame and with no CAD model at test time. Oriented bounding boxes play a supporting role. Smear stretches a moving object's image footprint along its motion direction, which is exactly where axis-aligned crops fail, so the system crops each object with a rotated box fitted to its streak before predicting coordinates.
Core Claim
The claim is that blur sensitivity in CAD-free pose estimation comes mainly from a wrong output representation, not a weak feature extractor. Changing the per-pixel target from a point to a time-indexed segment in canonical space should keep pose accuracy roughly flat as blur length grows. Standard single-coordinate heads trained on the same blurred data should degrade. The claim fails if a conventional head, trained on the same physically rendered blur with the mid-exposure pose as its target, matches the segment head. The proposal is organised around that comparison.
The image formation we assume is the standard exposure integral used in forward-modelled deblurring, applied to the object rather than the camera. Write $T(\tau)$ for the object-to-camera similarity transform at normalised exposure time $\tau \in [0,1]$, interpolated geodesically between $T_0$ and $T_1$ with shared scale $s$. Write $c_u^{k}$ for the predicted canonical coordinate seen by pixel $u$ at time $\tau_k \in \{0, \tfrac12, 1\}$, and $\alpha_u^{k}$ for the predicted probability that the object covered $u$ at that time. Write $\pi$ for camera projection and $d_u$ for measured depth. The solver minimises
$$ \sum_u \sum_k \alpha_u^{k}\,\big\| \pi\big(T(\tau_k)\, s\, c_u^{k}\big) - u \big\|^2 \;+\; \lambda \sum_u \alpha_u^{1/2}\big(d_u - z\big(T(\tfrac12)\, s\, c_u^{1/2}\big)\big)^2 , $$
inside RANSAC, the robust random-sampling scheme NOCS already uses for its Umeyama alignment. The first term ties each time slice to the pixel ray. The second uses depth only at mid-exposure. This choice is deliberate: depth sensors also smear under motion, and we treat depth as a single time-averaged measurement rather than trusting it at every instant. Setting $T_0 = T_1$ recovers an ordinary single-pose fit. The method should therefore never be worse than its static counterpart on sharp frames, and that is checked directly.
Relation to Prior Work
The category-level machinery is supplied by two papers. Normalized Object Coordinate Space defines the shared unit-cube category frame. It regresses per-pixel canonical coordinates on top of axis-aligned Mask R-CNN proposals and fits pose with Umeyama plus RANSAC. IST-Net shows that explicit category templates are unnecessary and that canonical-mapping supervision does the work. We keep both insights, and we reuse IST-Net's template-free design as our backbone. What we change is the target. Both papers predict one coordinate per pixel and were evaluated only on static frames. Neither has a mechanism by which a smeared pixel could be represented correctly.
The physics comes from forward blur modelling. BAD-Gaussians simulates exposure by averaging renders along a spline camera trajectory and optimises scene and trajectory jointly. BARD-GS separates camera blur from object blur in dynamic scenes. Kinematics-GS lets Gaussian primitives stretch only along their velocity. These papers motivate the proposal: they show that modelling the integral, rather than undoing it, yields sharp geometry. They also constrain it. All three are per-scene, multi-view optimisations taking tens of minutes, and none reasons about object categories, canonical frames or boxes. We move the integral into a feed-forward, single-frame, category-level predictor. Parametric Object Motion from Blur is the closest classical precedent for reading motion from a single blurred image. It recovers 2D parametric motion from oriented blur kernels on isolated targets, whereas we recover 3D pose and velocity in a category frame.
We deliberately do not claim that blur direction reveals object orientation, which is a tempting reading of the oriented-detection link. Blur direction reflects the relative motion between object and camera. A ship can drift sideways, and camera shake blurs every object the same way. The oriented box therefore encodes the object's footprint, meaning object plus streak, and not its heading. The oriented-detection literature supplies the cropping tools. RQFormer samples features strictly inside rotated regions through rotated RoI attention. It also notes that this sampling relies on object extremities that blur erodes, a weakness our footprint-level box sidesteps by enclosing the streak rather than the edges. EARL assigns training positives with orientation-aligned ellipses, which fits the elongated, soft-edged shape of a streak. Finding NeMO reports that mask proxies merge adjacent instances and names direct oriented-box prediction as open, which motivates using boxes rather than masks for crowded moving objects.
Research Program
The training data come from the synthetic CAMERA composites introduced with NOCS. Because those objects are rendered, we can render $K$ sub-exposure frames under sampled object and camera motions and average them in linear intensity. Each pixel then gets exact ground-truth canonical coordinates and coverage at every sub-time. Motions are sampled to include constant velocity, rotation about arbitrary axes, and camera shake. GeoSyn (Controllable Blur Data Augmentation Using 3D-Aware Motion Estimation) shows that 2D kernels miss real 3D blur structure, so all blur is produced by 3D integration rather than kernel convolution. CAD models are used only to manufacture training data, as in NOCS. Nothing object-specific is needed at test time.
The architecture has three parts. The first is a rotated-box detector, initialised from an RQFormer-style decoder and trained with EARL-style elliptical assignment, which encloses each object's full streak. The second is a coordinate head on rotated-RoI features. It outputs three canonical coordinate maps and three coverage maps, trained with coverage-weighted L1 on coordinates and binary cross-entropy on coverage. The third is the differentiable solver above, which also allows an end-to-end pose loss in a second training stage. Inference cost is one extra pair of output channels per time slice plus a slightly larger solve. We expect speed to stay near IST-Net's interactive regime, but we will measure it rather than assume it.
We will also study when the representation is identifiable. With pure camera shake, all objects share one motion, and coupling them across objects in a frame should help. With rotationally symmetric categories such as bowls, cans and bottles, rotation about the symmetry axis is unobservable in both pose and velocity. NOCS already handles this ambiguity for pose, and we will report velocity only for its observable components.
Baselines
The do-nothing reference is IST-Net trained on clean CAMERA plus REAL275 and run unchanged on blurred test frames. On clean REAL275, IST-Net reports 76.6% on the 3D IoU at 75% metric. NOCS reports 76.4% mAP at 3D IoU 50%. No reported result found on Lacuna for any category-level method on blurred inputs, so the drop under blur is itself a first measurement.
The strongest competitor is IST-Net retrained on exactly our blur-rendered CAMERA data, using the same backbone, schedule and tuning budget, with the mid-exposure canonical coordinate as its single target. This model sees the same physics during training and differs only in the output representation and solver. Beating it is the central test. We will count it as beaten only if our method gains on 5°5cm and 3D IoU 75 at medium and long blur lengths, with confidence intervals over three seeds that do not overlap, while staying within noise of it on sharp frames.
The second competitor is restore-then-estimate. GeoSyn-trained FFTformer is run on every frame, then clean IST-Net is applied. On RealBlur-J, GeoSyn reports FFTformer reaching 33.68 dB PSNR with its augmentation, up from 32.62 dB without. This is the strongest single-image deblurrer with a number in the file. We fine-tune it on our blurred CAMERA renders so it gets the same data access, and we report end-to-end latency alongside accuracy.
Two ablations isolate the key ingredient. One keeps our three-slice head but drops the coverage maps, which tests whether modelling partial occupancy at streak fringes matters. The other swaps the rotated-box crop for an axis-aligned box and for a mask crop with the coordinate head held fixed. This measures what the oriented-detection component actually buys. We expect any gain to appear mainly in scenes with several adjacent moving objects.
Why not just…?
Why not just deblur the frame first and run an existing pose estimator? This is a real threat for mild blur, and the restore-then-estimate baseline is in the study for that reason. The case against it is that single-image deblurring is ill-posed and can hallucinate edges. BAD-Gaussians motivates forward modelling over pre-deblurring on exactly these grounds, and fabricated edges are what correspondence and coordinate regressors latch onto. Deblurring also discards the velocity information we recover. If restoration matches our pose accuracy at all blur lengths, the pose claim fails and only the velocity output remains as a contribution. We will report that outcome plainly.
Why not just train the existing estimator with blur augmentation? This is the most dangerous alternative and the main comparison in the Baselines section. Its weakness in principle is that a single coordinate cannot describe a pixel that saw several surface points, so the best a single-coordinate regressor can do is a blur-dependent average that biases the Umeyama fit. Whether that bias matters in practice is an empirical question. The blur-length sweep settles it. If the gap stays flat as streaks lengthen, the representational argument is wrong.
Why not just rely on depth, which IST-Net already uses? Depth from active sensors is also corrupted by motion, and our objective already uses depth, but only as a mid-exposure average. To test whether a point-cloud method alone suffices, we will run a depth-only variant of IST-Net on rendered depth that is either blurred or kept sharp. If sharp-depth IST-Net closes the gap on its own, the honest conclusion is that the problem is sensor timing and not representation, and we will say so.
Why not just track across frames and get velocity by finite differences? Finite differences of per-frame poses inherit each frame's blur error and cannot resolve motion within the exposure. Our per-frame velocity can serve as an extra measurement inside a tracker. We will compare a constant-velocity filter on mid-exposure baseline poses against the same filter fed our two-endpoint estimates.
Proposed Experiments
We evaluate on three test sets with increasing realism. The first is held-out blur-rendered CAMERA, where ground truth is exact at every sub-time. The second is REAL275-Blur, built from REAL275 sequences by interpolating between consecutive annotated frames and averaging them. Its blur is approximate and caused by camera motion, which we state as a limitation. The third is a small newly captured set using a synchronised long-exposure and short-exposure camera pair, following the dual-GoPro protocol of BARD-GS. Reference poses come from the sharp camera with manual verification. Metrics are the standard REAL275 3D IoU at 50% and 75%, 5°5cm and 10°5cm, plus angular and linear velocity error against sub-time ground truth on CAMERA and against finite differences of sharp-camera poses on the captured set.
The decisive check is a sweep over streak length in pixels. Every method's accuracy is plotted against streak length, and the gap between our method and the blur-retrained IST-Net is tracked as streaks grow. Exposure physics predicts that the gap should widen with streak length: the spread of surface points under a pixel grows linearly with motion during exposure, and so does the bias of the averaged coordinate. A constant or vanishing gap falsifies the core claim. A second check is zero blur, where our method must match clean IST-Net within seed noise on REAL275, since the solver reduces to the static fit. A third is the crop ablation on scenes with several moving objects, which determines whether oriented boxes earn their place in the system or should be dropped in favour of axis-aligned crops.
References
- Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation.
- IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformation
- BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting
- BARD-GS: Blur-Aware Reconstruction of Dynamic Scenes via Gaussian Splatting
- Kinematics-Driven Gaussian Shape Deformation for Blurry Monocular Dynamic Scenes
- Parametric Object Motion from Blur
- RQFormer: Rotated Query Transformer for end-to-end oriented object detection
- EARL: An Elliptical Distribution Aided Adaptive Rotation Label Assignment for Oriented Object Detection in Remote Sensing Images
- Finding NeMO: A Geometry-Aware Representation of Template Views for Few-Shot Perception
- Controllable Blur Data Augmentation Using 3D-Aware Motion Estimation