Robots need the 3D position and orientation ("6D pose") of objects they grasp. Most methods get it either from an exact 3D CAD model of the object or by matching image features against a few reference photos. Glass breaks both routes. Depth sensors return holes or the depth of the table behind the glass. The RGB "texture" on a glass is mostly the background seen through it, so feature matching ends up matching the background. One signal does come from the object's own surface: the small fraction of light that reflects off glass is linearly polarized, and the angle of that polarization is set by the orientation of the surface. This note proposes estimating the pose of an unseen transparent object, with no CAD model, from a short posed reference sweep and a polarization camera. The method keeps only the polarization angle, written as a complex number of modulus one, and fits pose by aligning these "phasors" in the way phased-array antennas align signal phases.
The key observation is one that the polarization-vision papers in this file do not exploit. If the polarization angle $\phi$ is written as the unit-modulus number $e^{j2\phi}$, the angle-doubled phase used in wireless beamforming, two separate nuisances drop out. The first is the classic $\pi$-ambiguity: a surface tilted toward or away from the viewer gives the same reading. The second is the $90°$ flip between light reflected off the surface and light transmitted through it, which becomes a sign on a real amplitude. Discarding the amplitude (the degree of polarization) also removes any need to know the glass's refractive index, because for specular reflection the angle depends only on the plane of incidence. What remains is a continuous, physically grounded matching problem. It can be solved with the alternating phase-extraction and circle-manifold tools developed for hybrid precoding.
Core Claim
For transparent objects, the per-pixel phasor $e^{j2\phi}$ from a polarization camera is a better matching signal for CAD-free pose than RGB features or completed depth. A pose estimator that fits a physical phasor model, in which the measurement is a signed object reflection plus a background bleed term, will beat both learned-normal pipelines and RGB reference matchers on rotation accuracy. It also needs no refractive-index assumption and no fine-tuning on transparent materials.
Concretely, at pixel $i$ of the query image the camera measures the complex linear-polarization value $P_i = \rho_i e^{j2\phi_i}$, where $\rho_i$ is the degree and $\phi_i$ the angle of linear polarization. We model
$$P_i \approx a_i\, u_i(R,t) + b_i, \qquad |u_i| = 1,$$
where $u_i(R,t) = e^{j2\theta_i(R,t)}$ is the phasor predicted from the image-plane azimuth $\theta_i$ of the object's surface normal under rotation $R$ and translation $t$. Here $a_i$ is a real per-pixel gain, positive where reflection dominates and negative where light transmitted through the object dominates. $b_i$ is the background polarization leaking through the glass. TransSfP treats this leakage, which it calls the "4% problem", with a learned confidence map (Transparent Shape from a Single View Polarization Image). Here it is an explicit additive term, estimated from the background pixels around the object's mask. Because $u_i$ is invariant to flipping the normal's sign and $a_i$ absorbs the $90°$ flip, the discrete ambiguity labelling that Polarimetric Normal Stereo solves by belief propagation over a discretized sphere becomes a smooth least-squares fit.
Research Program
The pipeline mirrors the reference-based model-free estimators. Like Gen6D and OnePose++, it assumes a short sweep of reference images with known camera poses, for example from a robot wrist or a marker board, and never a CAD model. It replaces their appearance features with polarization phasors.
The reference stage builds a lightweight object representation. Transparent objects are featureless, so structure-from-motion of the kind OnePose relies on fails, but their silhouettes are visible. We segment each reference view with an open-world segmenter, as POPE does with SAM (POPE), carve a visual hull, and give each surface point a unit normal. Each normal starts from the hull and is then refined by fitting the multi-view phasor model. This is a small unit-sphere-constrained problem per point, solved with a few Riemannian gradient steps in the style of MO-AltMin (Alternating Minimization Algorithms for Hybrid Precoding). Across views, the stereo-style disambiguation shown by Polarimetric Normal Stereo arises naturally. Visual hulls miss concavities such as cup interiors, and we will report those regions separately rather than hide them.
The query stage has three steps. The first is detection with the same segmenter. The second is coarse retrieval of the nearest reference view by phasor correlation. This step uses an exact equivariance: rotating the camera by $\alpha$ about its optical axis rotates the image and multiplies every phasor by $e^{-j2\alpha}$, so the in-plane angle has a closed-form phase-extraction solution, the same operation as $\arg(\cdot)$ in PE-AltMin. The third is refinement by alternating minimization. With pose fixed, each $a_i$ has the closed form $\mathrm{Re}(\bar{u}_i(P_i - b_i))$. With gains fixed, $R$ takes a Riemannian step on the rotation group and $t$ a Euclidean step, driven by the phasor residual plus a silhouette-agreement term. Translation along the viewing direction comes from silhouette scale and from active-depth pixels on the rim where they are valid. Phase carries little depth information, so we expect most of the gain to show up in rotation and will report rotation and translation errors separately.
The three directions play different roles. CAD-free pose sets the problem and the evaluation protocol. Transparent perception supplies the regime and the physics. The unit-modulus literature supplies the representation and solvers: phasor alignment, closed-form phase extraction, and circle and sphere manifold steps (Manifold Optimization Based Multi-user Rate Maximization). We deliberately do not claim that heavier machinery is needed. Extreme Point Pursuit does cover rotation and sphere constraints, but rotation here has only three degrees of freedom, and exhaustive search is a serious competitor. EXPP enters only as a fallback if the alternating solver shows local-minimum failures that grid search does not.
Relation to Prior Work
Two lines of work motivate the gap. The first is transparent-object pose methods, which remove instance CAD models but stay tied to categories. TransNet completes depth, estimates normals, and regresses rotation axes followed by Gram-Schmidt orthogonalization. StereoPose predicts front and back-view canonical coordinates from stereo, and NOCS provides the canonical-space idea both build on. All three need training on the object's category. We need only a reference sweep of the specific object.
The second line is reference-based model-free estimators: Gen6D, POPE, and OnePose++. They generalize to new objects but match RGB appearance, which on glass is dominated by whatever lies behind it. ClearPose shows the result for depth-completion pipelines, which drop to 13% ADD-S under translucent covers.
Two works constrain the design. HouseCat6D supplies polarization data alongside transparent objects and notes that current estimators do not use polarization constraints. TransSfP shows that physics priors beat pure learning for normals, with a mean angular error of 16.29° against 19.83° for DeepSfP, but it assumes $n=1.52$ and stops at normals. We use only the index-free phase and go all the way to pose.
The closest model-free transparent pose method is Object Pose Estimation Using Implicit Representation for Transparent Objects. It renders NeRF references inside a render-and-compare loop and reaches high accuracy, but it needs material-specific fine-tuning and many rendered hypotheses. We replace rendering with a closed-form phasor prediction from the object's normals.
Baselines
The naive reference point is the same reference-sweep protocol fed to Gen6D and POPE on RGB alone, with no polarization and no depth repair. Both run with their released matching modules and the same reference views. This measures how badly appearance matching fails on glass. Any useful method must clear it on rotation error by a wide margin.
The "repair the depth, then treat it as opaque" baseline completes depth with TODE-Trans (TODE-Trans) and then runs a reference-based RGB-D estimator. The relevant prior evidence is the 44.3% and 46.9% 3D25 that SPD and SGPA obtain in StereoPose's comparison. We rerun it on our split with the same references and masks.
The strongest existing method is NeRF render-and-compare (Object Pose Estimation Using Implicit Representation for Transparent Objects). On HouseCat6D it reports 78.7% at the 10°/5cm threshold with fine-tuning. We train its NeRF from exactly our reference sweep. We run it both with and without its transparent-material fine-tuning, and we cap its hypothesis count so its wall-clock time is reported next to ours. Beating it means matching or exceeding its 10°/5cm accuracy without fine-tuning, at markedly lower latency, or exceeding it outright on the transparent-glassware subset. A meaningful secondary bar is a FoundationPose-style tracker given our visual-hull mesh and FoundationStereo depth, the stack used by DeLTa.
Three ablations isolate the key ingredient. The first swaps the phasor residual for an angular residual on $\phi$ with the ambiguities resolved by nearest-branch selection. The second swaps phasors for learned normals from a TransSfP-style network, registered with the same solver. The third replaces the alternating solver with exhaustive search over a fine rotation grid using the same score. If the third ablation matches our accuracy, we will adopt grid search and the contribution rests on the signal alone.
Why not just…?
Why not just complete the depth and run an opaque-object pose estimator? Prior evidence is unfavorable. ClearPose reports drops to 13% ADD-S under translucent covers for such pipelines, and StereoPose's depth-completion baselines reach under half its bottle accuracy. Depth completion also hallucinates geometry from RGB, which carries the background-bleed problem. We still run this baseline, because on clean, isolated glassware it may be competitive on translation, and we expect it to lose mainly on rotation.
Why not just predict surface normals with a network and register those? This is the most serious threat. TransSfP already gives normals at 16.29° mean error, and registering a predicted normal map against the hull needs no phasor model. Its weaknesses are a fixed refractive-index assumption, training on specific objects, and a learned treatment of background leakage. The learned-normal ablation runs exactly this with our solver. If it matches the phasor model on HouseCat6D glassware, the honest conclusion is that the explicit physics model buys robustness only under background change, and the background-swap test below decides that.
Why not just use NeRF render-and-compare, which already handles refraction? It is accurate but slow, and its authors needed transparent-specific fine-tuning, which did not transfer to shiny metal on DIMO. Our comparison holds the reference data fixed and reports accuracy against latency. If NeRF wins on accuracy at equal compute, we will say so. Our expected advantage is no fine-tuning and phase prediction in a single pass.
Why not just grid-search rotation instead of using manifold optimization? For one object's rotation this may well suffice, which is why it is an ablation rather than a straw man. The manifold machinery earns its place only in the multi-view normal refinement, where thousands of coupled unit-sphere variables preclude enumeration.
Proposed Experiments
The main benchmark is HouseCat6D, the only dataset in the file that combines polarization imaging with transparent glassware and accurate poses (annotation error of 1.35 to 1.74 mm). Its categories include glass, cutlery, and cups. We build an instance-level, model-free protocol: references come from one scene's sweep of an object, and queries come from every other scene containing it. Metrics are the 10°/5cm and 10°/10cm accuracies, ADD-S, and rotation and translation errors reported separately. Symmetry-aware rotation error follows NOCS, since many glasses are surfaces of revolution with an unobservable spin.
Before any pose experiment, we check the measurement model. On the TransSfP real set, with laser-scanned normals, we measure how well $a_i u_i + b_i$ with ground-truth normals explains the observed phasors, and how often the sign of $a_i$ flips. We found no azimuth-only number on Lacuna for TransSfP, so we compute both TransSfP's azimuth error and ours.
The decisive tests are two stress tests on HouseCat6D. The first is a background-change test: references and queries are taken against visibly different backgrounds, a setting where appearance matchers should collapse and phasor matching should not. The second is a translucency test that bins queries by how much of the object overlaps other transparent objects, echoing ClearPose's multi-layer failure. The proposal succeeds if the phasor method's rotation accuracy holds up under background change and the RGB and learned-normal variants do not. It fails, and we will report that, if learned normals match it there too.
References
- Transparent Shape from a Single View Polarization Image
- Polarimetric Normal Stereo
- Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images
- OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models
- POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference
- Alternating Minimization Algorithms for Hybrid Precoding in Millimeter Wave MIMO Systems
- Manifold Optimization Based Multi-user Rate Maximization Aided by Intelligent Reflecting Surface
- Extreme Point Pursuit - Part I: A Framework for Constant Modulus Optimization
- TransNet: Category-Level Transparent Object Pose Estimation
- StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS
- Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation.
- ClearPose: Large-scale Transparent Object Dataset and Benchmark
- HouseCat6D - A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios
- Object Pose Estimation Using Implicit Representation for Transparent Objects
- TODE-Trans: Transparent Object Depth Estimation with Transformer
- DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulation