A robot grasping a square block from above can succeed at four yaw angles spaced 90° apart. For a long rectangular bar there are two such angles, and for a mug there is only one. Equivariant reinforcement learning builds rotations into the policy, so a lesson learned at one orientation transfers to every rotated copy of the scene. That transfer only covers rotations of the whole scene about the camera's center. The rotational symmetry of a single object sitting off-center among clutter is invisible to it. The agent therefore learns the value of each of the four equivalent square-block grasps separately. When it outputs grasp yaw as a continuous angle through a Gaussian policy, it also runs into the wrap-around cliff at ±π, where a 1° physical error looks like a 359° error.
We propose to detect each object's local rotational symmetry from the image: its rotation center and fold order $k$, meaning how many times per turn it maps onto itself. We then use that detection to tie together the agent's value estimates and action distribution across the $k$ equivalent angles. The angular action is represented on the reduced circle of period $2\pi/k$, using the harmonic encoding $e^{jk\theta}$. Oriented object detection already showed that this encoding removes boundary discontinuities, because those discontinuities come from the object's own symmetry. A consistency test driven by reward prediction error lets the agent overrule the detector when it is wrong. In two sentences: global equivariance cannot exploit the symmetry of individual objects in a cluttered scene, but a symmetry detector can name it, and naming it lets one experience update all $k$ equivalent actions while removing the angle wrap-around problem. The detector supplies a prior, and the agent's own reward prediction errors check it.
Core Claim
The claim has two parts. First, in cluttered, globally asymmetric tabletop scenes, tying the grasp and placement yaw of each object across its detected local rotational fold order measurably improves sample efficiency over strictly equivariant actor-critic methods. Second, it keeps directed actions intact, such as pulling a drawer or pushing along a heading, because those receive $k=1$. The gain should grow with the average fold order of the objects and vanish for asymmetric ones. That dose-response is the signature that the mechanism, rather than extra capacity, drives any improvement.
Where This Sits
The proposal draws on three bodies of work, each playing a different role.
Equivariant reinforcement learning supplies the base agent. SO(2)-Equivariant Reinforcement Learning shows that for rotation-symmetric tasks the optimal critic, the network estimating action values, must be invariant, and the optimal actor, the network choosing actions, must be equivariant. Its Equivariant SAC reaches peak Block Pulling performance in roughly 2,500 steps, where augmentation baselines RAD and DrQ need over 10,000. Equivariant Reinforcement Learning under Partial Observability extends this to agents with memory on BulletArm. Both enforce symmetry only about the global image frame and parameterize continuous angles as unbounded Gaussians. We keep their architecture and add a local, per-object quotient on the angular action.
Two works motivate and constrain the proposal. Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments finds that forcing global symmetry where it does not hold poisons the Bellman backup, the step where the critic updates its estimate from its own next-step prediction. It detects violations by comparing an equivariant and an unconstrained forward model, and falls back to an unconstrained critic where they disagree. This is a warning that wrongly asserted symmetry is costly. It is also the template for our verification test. PERL only subtracts symmetry, whereas we add local symmetry that the global group never contained. Hierarchical Equivariant Policy via Frame Transfer anchors a local frame for its low-level policy but transfers only translation, leaving orientation unanchored. The detected rotation center and axis phase provide exactly that missing rotational anchor.
Symmetry detection and angular encoding supply the method. Axis-level Symmetry Detection with Group-Equivariant Representation predicts explicit rotation centers with fold orders rather than fuzzy heatmaps. It reports rotation center sAP of 40.0% and fold sAP of 28.9% on DENDI, and notes frequent 2-fold/4-fold confusion. This error mode shapes our design: the detector must be treated as uncertain. Rethinking Boundary Discontinuity Problem for Oriented Object Detection argues that angle discontinuities in box regression arise from the box's own symmetry, and encodes angles as $e^{j\omega\theta}$ with $\omega$ fixed per shape class. Adding this to KFIoU lifts HRSC2016 $AP_{75}$ from 62.95 to 87.77. We make $\omega$ a per-object quantity inferred from images and move the encoding from passive regression into a policy and a critic.
The same paper warns that $\omega>1$ destroys directional heading. Arbitrary-Oriented Object Detection with Circular Smooth Label needed a separate heading classifier (OHDet) for this reason. We handle heading by letting the task-conditioned actor keep a $k=1$ channel for directed actions.
Research Program
The agent is Equivariant SAC on top-down images with actions $(x, y, z, \theta, \text{gripper})$. A symmetry module runs on the same image. It is the axis-level detector, pretrained on DENDI and fine-tuned on rendered simulator scenes. At each candidate action location it outputs a distribution $w_k(s,x)$ over fold orders $k \in \{1,2,3,4,6,\infty\}$ and a phase $\phi$, the angle of a detected reflection axis. The case $k=\infty$ covers circular objects such as cylinders. Here $s$ is the observation and $x$ is the action's image position.
The critic is replaced by a symmetry-tied version. It averages the base equivariant critic $Q$ over the angles that the detected symmetry declares equivalent:
$$Q_{\text{tied}}(s,x,\theta) = \sum_k w_k(s,x)\,\frac{1}{k}\sum_{m=0}^{k-1} Q\!\left(s,x,\theta + \tfrac{2\pi m}{k}\right).$$
As a result, a single transition at angle $\theta$ updates the value at all $k$ equivalent angles. The actor's yaw is drawn from a von Mises distribution, a Gaussian-like distribution that lives on a circle, placed on the reduced circle: $\theta = \phi + \psi/k$ with $\psi \sim \text{vM}(\mu, \kappa)$. Here $\mu$ and $\kappa$ are the location and concentration output by the actor. The critic's angle input is encoded as $(\cos k(\theta-\phi), \sin k(\theta-\phi))$. This removes the ±π cliff and makes the $k$ equivalent grasps a single mode rather than $k$ competing ones. When $k$ is uncertain, the actor samples $k$ from $w_k$.
Verification borrows PERL's logic but applies it locally. For each object region, the agent keeps a running estimate of how much the untied critic's values disagree across the $k$ angles that the detector claims are equivalent, measured on the agent's own replay data. If the disagreement exceeds what the critic's own error would explain, the posterior on $k$ shifts toward divisors of $k$. For example, a false 4-fold on a rectangular bar collapses to 2-fold. The detector thus gives a strong prior early, while experience can correct it later.
Directed tasks, and task phases such as pulling, pushing, or opening, are conditioned to $k=1$ by the goal specification. This keeps heading information intact. We will also test a learned gate that decides between tied and directed behavior.
Baselines
The naive reference point is Equivariant SAC with a Gaussian yaw regressed as a raw scalar, exactly as in SO(2)-Equivariant Reinforcement Learning. This is the configuration whose wrap-around and multimodality problems we target.
The strongest existing method is that same Equivariant SAC with the cheapest possible fix for wrap-around: a $(\cos\theta, \sin\theta)$ yaw output. Its reported result is peak Block Pulling performance in about 2,500 steps against over 10,000 for RAD and DrQ. On our cluttered multi-object variants there is no reported result found on Lacuna for any method, so we will rerun it ourselves.
We also include the categorical rotation-bin Q-map from the same paper's Equivariant DQN. It handles periodicity and multimodality by classification, in the spirit of Circular Smooth Label. It is the most serious competitor because it needs no detector. PE-SAC from Partially Equivariant Reinforcement Learning represents the best existing treatment of symmetry mismatch. RAD/DrQ augmentation represents the non-equivariant family.
All methods share the encoder backbone, replay size, environment steps, and a matched hyperparameter search of 20 trials per method. Detector pretraining compute is reported separately and also granted to baselines as a pretrained visual encoder.
Three ablations isolate the key ingredient. The first gives ground-truth $k$ from the simulator (oracle fold), which sets the ceiling. The second keeps the harmonic encoding but drops critic tying, which separates the discontinuity fix from sample sharing. The third keeps tying but disables verification, which measures what detector errors cost.
Beating the strongest baseline means reaching 90% of its final success in at most half its environment steps on the multi-object tasks. The gain must hold across 5 seeds and be absent on $k=1$ objects. If tying helps asymmetric objects too, the gain is not coming from symmetry.
Why not just…?
Why not just use rotation augmentation? The equivariant RL results already show augmentation trailing hard-coded symmetry by roughly 4× on Block Pulling, and failing outright on Object Picking and Drawer Opening within budget (SO(2)-Equivariant Reinforcement Learning). Augmentation also has no mechanism for local symmetry: rotating the whole image does not tell the agent that one off-center block is 4-fold symmetric. We include RAD and DrQ, but expect them to be the weakest.
Why not just discretize yaw into bins, as in Equivariant DQN or Circular Smooth Label? This is the real threat. Bins fix the wrap-around and can represent $k$ modes, so the entire case rests on sample sharing: bins must learn each equivalent angle's value from separate experience, while tying updates all of them at once. If the categorical baseline matches our oracle-fold ablation in sample efficiency, the idea is refuted. We would then report that result rather than argue around it. The fold-order dose-response experiment is designed to settle this question.
Why not just rely on global equivariance? An equivariant network does output $k$-symmetric values for a $k$-symmetric object, but only if that object sits at the image center with nothing asymmetric around it. Our cluttered, off-center scenes are built to break this condition. A diagnostic will measure how far the base critic's values vary across the $k$ equivalent angles, confirming the gap exists before claiming to close it.
Why not just discover $k$ from the critic alone, without a vision detector? Our verification step already does this, so the question is whether the detector adds anything. A "critic-only discovery" variant starts from a uniform prior on $k$. If it matches the full system, the detector is unnecessary. We expect it to lag early, because inferring $k$ from values requires exactly the repeated experience that tying is meant to save.
Proposed Experiments
The experiments use BulletArm (BulletArm) closed-loop tasks: Block Picking, Block Stacking, House Building, the 11-object Object Picking, and Drawer Opening and Block Pulling as directed ($k=1$) controls. Each is extended with a cluttered variant placing 3–6 objects of mixed fold orders at off-center positions, plus a PERL-style fixed obstacle that breaks global symmetry.
The metrics are steps to 90% of final success, final success, zero-shot success on unseen orientations and unseen object shapes, and the rate of gradient-norm spikes as a direct probe of wrap-around instability.
The decisive checks are three. The first is dose-response: the gap over the strongest baseline should rise with the fold order $k$ of the objects. The second compares the detector-driven system to the oracle-fold ablation, with detector fold accuracy on sim renders reported alongside, to price in the 2/4-fold confusion noted in the DENDI results. The third tests directed tasks, where the method must not fall below the baselines, showing that the $k=1$ channel preserves heading.
A secondary offline check re-evaluates the fine-tuned detector on DENDI against the reported 28.9% fold sAP. This confirms that simulator fine-tuning did not destroy its real-image ability before any transfer to a physical UR5 arm.
References
- $\mathrm{SO}(2)$-Equivariant Reinforcement Learning
- Equivariant Reinforcement Learning under Partial Observability
- Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments
- Hierarchical Equivariant Policy via Frame Transfer
- Axis-level Symmetry Detection with Group-Equivariant Representation
- Rethinking Boundary Discontinuity Problem for Oriented Object Detection
- Arbitrary-Oriented Object Detection with Circular Smooth Label
- BulletArm: An Open-Source Robotic Manipulation Benchmark and Learning Framework