Efficient Continuous Group Convolutions for Local SE(3) Equivariance in 3D Point Clouds

Lisa Weijler, Pedro Hermosilla

The Challenge of Rotation in 3D Vision

In 3D computer vision, objects rarely appear in a "canonical" orientation. While standard Convolutional Neural Networks (CNNs) are naturally translation-equivariant—recognizing patterns regardless of their spatial location—they lack inherent rotation-equivariance. This poses a significant hurdle for tasks like object classification and scene understanding where orientations are arbitrary.

Existing solutions typically fall into two categories, both with significant drawbacks:

  1. Global Equivariance: These models ensure that if an entire scene rotates, the output rotates accordingly. However, they fail when individual components within a scene rotate independently, such as a person waving their arms or a door opening.
  2. Computational Complexity: True group convolutions on the SE(3) group (the set of all rotations and translations) are mathematically "6D" operations. This makes them memory-intensive and slow, often rendering them impractical for large-scale 3D scans.
Global vs. Local Equivariance: Global models fail when parts of a scene rotate independently, whereas local models handle articulated movement.

Proposed Methodology: Efficient Local Group Convolutions

The authors introduce a highly efficient continuous group convolution layer designed to achieve local SE(3) equivariance with a computational footprint nearly identical to standard, non-equivariant convolutions. The process follows a five-step pipeline:

The 5-step pipeline for efficient group convolutions: 1) Sample receptive field, 2) PCA, 3) Frame building, 4) Stochastic sampling, 5) Group convolution.
  • Local Reference Frames (LRF): For every point in a cloud, the model computes a local coordinate system using Principal Component Analysis (PCA) on its neighbors.
  • Frame Building: Because PCA has sign ambiguity, it results in a "frame" of four possible rotations. The convolution is performed over this finite, point-specific frame rather than the entire infinite group of rotations.
  • Stochastic Approximation: To maximize efficiency, the model randomly samples only one or two rotations from the frame during training. This drastically reduces memory usage and computation. Surprisingly, this stochasticity acts as a regularizer, enhancing robustness to noise and estimation errors during testing.
  • Continuous Kernels: The kernel is implemented as a small Multi-Layer Perceptron (MLP), allowing it to process unstructured point clouds directly without a rigid grid.

Experimental Results and Benchmarks

The approach was evaluated across three distinct tasks to test its versatility and robustness:

  • Object Classification (ModelNet40): Under random rotations, the proposed method achieved 86.9% accuracy, whereas a standard non-equivariant model plummeted to 12.3%.
  • Part Segmentation (DFAUST & PosePrior): This served as a "stress test" for local equivariance because human limbs move independently. The method achieved 95.0 mAcc, significantly outperforming global equivariant methods like Vector Neurons (63.1 mAcc).
  • Scene Understanding (ScanNet): Tested on large-scale indoor scans to demonstrate real-world applicability.

Efficiency Gains

Compared to other local equivariant methods like E2PN, this approach is approximately 13x faster (581 FPS vs 45 FPS) and uses 32x less memory (37 MB vs 1211 MB) for a single convolution layer. By sampling only one rotation during training, the model achieves rotation robustness almost for "free."

Conclusion

By restricting equivariance to a local receptive field and utilizing a clever stochastic sampling strategy, the authors have developed a method that handles articulated objects and complex scenes far better than global models. It generalizes remarkably well to out-of-distribution poses—such as a person doing a handstand—that were never seen during training. While the method relies on the stability of local PCA, its efficiency and performance mark a significant step forward for practical, rotation-robust 3D computer vision.

Summary updated

Abstract

Extending the translation equivariance property of convolutional neural networks to larger symmetry groups has been shown to reduce sample complexity and enable more discriminative feature learning. Further, exploiting additional symmetries facilitates greater weight sharing than standard convolutions, leading to an enhanced network expressivity without an increase in parameter count. However, extending the equivariant properties of a convolution layer comes at a computational cost. In particular, for 3D data, expanding equivariance to the SE(3) group (rotation and translation) results in a 6D convolution operation, which is not tractable for larger data samples such as 3D scene scans. While efforts have been made to develop efficient SE(3) equivariant networks, existing approaches rely on discretization or only introduce global rotation equivariance. This limits their applicability to point clouds representing a scene composed of multiple objects. This work presents an efficient, continuous, and local SE(3) equivariant convolution layer for point cloud processing based on general group convolution and local reference frames. Our experiments show that our approach achieves competitive or superior performance across a range of datasets and tasks, including object classification and semantic segmentation, with negligible computational overhead. The code for our implementation is available at this repository.
BibTeX
@inproceedings{weijler2025efficient,
  author = {Lisa Weijler and Pedro Hermosilla},
  title = {{Efficient Continuous Group Convolutions for Local SE(3) Equivariance in 3D Point Clouds}},
  booktitle = {3DV},
  year = {2025},
  url = {https://openreview.net/forum?id=mLWpFP1eS7},
}
Figure