When a robot looks at a symmetric object, several different 3D rotations can produce exactly the same image. A box seen from the side looks identical after a half-turn. A mug whose handle is hidden looks the same at every spin about its axis. An honest pose estimator should therefore output a distribution with several peaks. The natural tool for that is a mixture model, meaning a weighted sum of simple bell-shaped distributions. On rotations, these mixtures are notoriously hard to train. A component can shrink its spread until the training likelihood blows up. Two components meant to cover distinct symmetric poses can drift together and merge. Practitioners patch this with hard floors on the spread, which damages calibration. Alternatively, they abandon compact mixtures for grids or diffusion samplers that are slower.
We propose a mixture over rotations whose components are not free. Every component is the same concentrated rotation distribution, copied onto a candidate symmetry of the object's CAD model. The network predicts, for each image, only one shared spread and a weight for each candidate symmetry. Because the copies share their spread and sit at fixed separations given by the geometry, the two known routes to collapse are removed by construction rather than clipped after the fact. The per-image weights still let the model express that a visible handle removes an ambiguity and a hidden one restores it. We will test this on the per-image ambiguity annotations of BOP-Distrib, where it must beat grid-based and diffusion-based distribution estimators on precision and recall while keeping the cost of a single-peak parametric head.
Core Claim
The mixture-collapse literature identifies two separate failure mechanisms.
The first is spread shrinkage. Archambeau et al. describe it as a feedback loop in which a component isolates a few points, narrows to fit them, loses responsibility for its neighbours, and narrows further until the likelihood diverges. Their follow-up work and Forero et al. tie this loop to components having independent, unconstrained covariances.
The second is "pile-up". Weak Separation in Mixture Models shows that when modes are weakly separated, maximum likelihood can drive the distance between component means to exactly zero. On one of their causal-inference benchmarks, JOBS II ($N=132$), this happens with 63% probability. Singularity Structures in Finite Mixtures shows that such degenerate configurations slow estimation rates from $n^{-1/2}$ to as slow as $n^{-1/8}$.
Both mechanisms need degrees of freedom that a symmetric object does not actually have. The copies of a pose under a symmetry have the same shape of uncertainty. They are also separated by a known rotation, not a learned one.
We therefore build on the Matrix Fisher distribution, a bell-shaped density over rotations $R$ parameterized by an unconstrained $3\times3$ matrix $F$. Mohlin et al. show that its negative log-likelihood is convex in $F$, which makes it stable to train, but they also note that it has only one peak. Our proposed density is
$$ p(R \mid x) = \frac{1}{c(F(x))} \sum_{g \in \mathcal{G}_{\text{obj}}} w_g(x)\, \exp\!\big(\mathrm{tr}\big((F(x)\,g)^\top R\big)\big). $$
In this formula, $x$ is the image crop and $F(x)$ is the shared Matrix Fisher parameter predicted by the network. $\mathcal{G}_{\text{obj}}$ is a finite library of candidate object-frame rotations computed from the CAD model, always including the identity. The weights $w_g(x)$ are per-image and lie on the probability simplex. $c(F)$ is the Matrix Fisher normalizer.
Two properties follow directly from the construction. The normalizer depends only on the singular values of $F$, and right-multiplying by a rotation $g$ does not change those singular values. So every component shares one normalizer, and the model costs essentially the same as a single Matrix Fisher head. Continuous ambiguities, such as the ring of poses around a cylinder's axis, are still carried by the anisotropic singular values of $F$, just as in the unimodal model.
Our claim is empirical and structural rather than a convexity theorem. We do not claim the full negative log-likelihood is convex. It is convex in the weights for fixed $F$, and that is all we assert. The claim is that removing independent spreads and learned separations eliminates the collapse modes that plague free rotation mixtures. Learned per-image weights then recover the occlusion-dependent ambiguity that global CAD symmetry groups miss.
Relation to Prior Work
Two papers motivate the gap directly. Deep Orientation Uncertainty Learning based on a Bingham Loss regresses a single Bingham distribution, another bell-shaped density on unit quaternions. It already needs a two-stage schedule to train stably, and it reports that mixtures of Bingham distributions are hard to balance. Mohlin et al. name multi-peak losses with good optimization properties as open future work.
The closest existing method is Deep Bingham Networks. It trains a free Bingham mixture with a relaxed winner-takes-all loss, in which the best-matching component receives most of the training signal. That loss prevents components from going entirely unused. It does not address spread singularities, and it learns component locations freely. Our model does the opposite: locations are fixed by geometry, spreads are tied, and only the weights are free.
In Euclidean dense regression, Varamesh and Tuytelaars show that mixture heads help. On COCO, human-pose AP rises from 46.4 to 52.3 with three components. However, those heads collapse unless the variance is forced to be at least 1. That result constrains our design by showing what the clamp alternative costs.
The regularized-likelihood line of work supplies analogues but not tools we reuse directly. It includes guaranteed-existence penalties on Fisher–Rao manifolds (Riemannian Optimization for Non-Centered Mixtures) and the $h$-lifted divergence (Risk Bounds via h-Lifted KL). Neither is formulated on rotation groups. We therefore use them only to shape a shrinkage-penalty ablation.
BOP-Distrib plays two roles. It supplies the candidate library, because its re-annotation pipeline precomputes elementary symmetry patterns on CAD vertices and intersects them over visible vertices. It also supplies the evaluation, because it shows that ambiguity is per-image, driven by viewpoint and occlusion, and not a fixed CAD property.
Research Program
Building the candidate library
For each object, we compute partial self-alignments. These are rotations that map a large fraction of the surface onto itself. We reuse the elementary-pattern computation of BOP-Distrib. This yields global symmetries and also partial ones, such as the mug's spin symmetry with the handle excluded. Continuous partial symmetries are discretized into a modest number of copies, for example 36 steps about the axis, and a low-concentration $F$ smooths between them. The library is capped at about 72 elements. It is precomputed once per object and requires only the CAD model, which T-LESS and YCB-V provide.
Training from single-pose labels
The model trains by maximum likelihood on single annotated poses, with no symmetry-aware target. When a disambiguating feature is visible, the labels are consistent, so the weights can concentrate on the correct element. When the feature is hidden, the labels scatter across the orbit, and maximum likelihood spreads the weights accordingly. We will verify that this learning actually happens rather than assume it.
The backbone and translation head follow SpyroPose's setup, and this study is scoped to rotation. We will also check that when the library contains only the identity, the model reduces exactly to Mohlin et al.'s Matrix Fisher head.
Collapse diagnostics
For every model that has free components, we will log three quantities during training:
- each component's maximum singular value, which tracks spread shrinkage;
- the minimum geodesic distance between the peaks of different components, which tracks pile-up;
- the fraction of runs whose loss diverges.
These make the collapse claim measurable rather than rhetorical.
Baselines
Unimodal Matrix Fisher. The naive reference is a single Matrix Fisher head (Mohlin et al.) with the same backbone. The file reports 8.17° median error on Pascal3D+ and 18.0° on ModelNet10-SO(3) for this head. It is the right floor because our model contains it as a special case. Beating it means higher distribution recall and log-likelihood on discretely symmetric T-LESS objects, with no loss on asymmetric ones.
Quotient Matrix Fisher with uniform orbit expansion. This is the most dangerous parametric baseline. We train a Matrix Fisher head with a loss that takes the minimum over the global symmetry group, as ShapeMatch-Loss does in PoseCNN. At test time, we spread its mass uniformly over the group. It uses the same CAD knowledge but cannot vary its weights per image. It therefore isolates our key ingredient, and beating it on BOP-Distrib's per-image precision is the central test.
Free Matrix Fisher mixture. This is a $K$-component mixture with independent $F_k$, trained with the relaxed winner-takes-all loss of Deep Bingham Networks. It is run in three variants: unclamped, with a singular-value floor in the style of the Varamesh clamp, and with a shrinkage penalty toward a pooled $F$. Together these represent the best existing way to fit multi-peak rotation densities.
Tying ablations. Two ablations separate which ingredient does the work. The first ties spreads but learns component locations. The second fixes locations to the library but leaves spreads independent.
Non-parametric estimators. The strongest distribution estimators are SpyroPose, which evaluates probabilities on a coarse-to-fine grid over poses, and LiePose-Diffusion as benchmarked in BOP-Distrib. BOP-Distrib reports that LiePose-Diffusion generally has higher recall than SpyroPose on real images. No reported numerical result was found on Lacuna for either method's T-LESS distribution precision or recall. We will therefore rerun both under BOP-Distrib's protocol.
Fairness protocol. All methods share the same training images, backbone, number of epochs, and an equal hyperparameter-search budget.
What counts as winning. Beating the non-parametric methods means matching or exceeding their recall at higher precision, with a single forward pass. SpyroPose's reported speeds are 53.5 FPS on GPU for rotation-only models and 16.1 FPS for full SE(3), the joint space of rotation and translation. On SYMSOL I, the best number in the file is an average log-likelihood of 5.11, from Equivariant Single View Pose Prediction.
Why not just…?
Why not just clamp the spread, as MDNs do? The Varamesh clamp works in 2D image coordinates. On rotations, uncertainty is direction-dependent: a floor on the singular values of $F$ inflates uncertainty along axes where the image is unambiguous. A floor also does nothing about pile-up, since the peak locations of clamped components can still merge. The clamped free-mixture baseline and the peak-distance diagnostic test both concerns directly.
Why not just use ShapeMatch or another symmetry-aware point loss? Such losses remove conflicting gradients, but they output a single pose. BOP-Distrib shows that single-pose estimators lose substantially under per-image distribution evaluation. It reports recall drops often of 10–30% across methods and large rank changes. We include the distributional version of this idea, the quotient-plus-uniform baseline, as the real threat. If it matches our model on BOP-Distrib precision, per-image weights add nothing, and we would report that.
Why not just use SpyroPose or diffusion? They avoid likelihood singularities entirely and may win on raw recall. Their costs are grid resolution and sampling time. There is also evidence that less-structured models degrade with little data: on 10% of SYMSOL, implicit models fall to −7.94 log-likelihood, versus 3.01 for a structured equivariant model (Equivariant Single View Pose Prediction). We will compare accuracy and speed, and add a reduced-data sweep, to see whether the CAD-derived structure pays off.
Why not just a free mixture with winner-takes-all training? Deep Bingham Networks shows this avoids unused components. It says nothing about spread singularities, and Gilitschenski et al. report that such mixtures are unstable to balance. The divergence rate and the tying ablations will show whether stability comes from tied spreads, fixed locations, or both.
Proposed Experiments
Main evaluation. On T-LESS and YCB-V with BOP-Distrib per-image annotations, we report distribution precision and recall, test negative log-likelihood of the annotated pose, and frames per second.
Calibration. We report the Expected Absolute Angular Deviation of Gilitschenski et al. against the realized angular error.
Synthetic check. On SYMSOL I and II, we compare average log-likelihood against the reported 5.11. We will use SYMSOL II's partially marked solids as a controlled test of view-dependent weights, if their construction behaves as we understand it.
Weights versus visibility. The first decisive check tests whether the learned weights track visibility. The entropy of $w(x)$ should correlate with the size of BOP-Distrib's per-image ground-truth set. It should also change when we synthetically occlude a disambiguating feature in held-out renders.
Collapse removal. The second decisive check tests whether tying removes collapse. Across seeds, the free mixtures should show shrinking peak separations or exploding singular values, while the orbit-tied model should show neither, at equal or better likelihood.
Library coverage. The third check probes the model's main weakness: a candidate library that misses an ambiguity. We will prune library elements and measure whether $F$ absorbs the missing mode by smearing, as the unimodal model does. This tells us how much of the gain depends on CAD preprocessing quality.
References
- On Convergence Problems of the EM Algorithm for Finite Gaussian Mixtures
- Robust Clustering Using Outlier-Sparsity Regularization
- Weak Separation in Mixture Models and Implications for Principal Stratification
- Singularity Structures and Impacts on Parameter Estimation in Finite Mixtures of Distributions
- Probabilistic Orientation Estimation with Matrix Fisher Distributions
- Deep Orientation Uncertainty Learning based on a Bingham Loss
- Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation
- Mixture Dense Regression for Object Detection and Human Pose Estimation
- Riemannian Optimization for Non-Centered Mixture of Scaled Gaussian Distributions
- Risk Bounds for Mixture Density Estimation on Compact Domains via the h-Lifted Kullback–Leibler Divergence
- BOP-Distrib: Revisiting 6D Pose Estimation Benchmarks for Better Evaluation under Visual Ambiguities
- PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
- SpyroPose: SE(3) Pyramids for Object Pose Distribution Estimation
- Equivariant Single View Pose Prediction Via Induced and Restriction Representations