Equivariant Networks for Pixelized Spheres

Mehran Shakerinava, Siamak Ravanbakhsh

Abstract

Pixelizations of Platonic solids such as the cube and icosahedron have been widely used to represent spherical data, from climate records to Cosmic Microwave Background maps. Platonic solids have well-known global symmetries. Once we pixelize each face of the solid, each face also possesses its own local symmetries in the form of Euclidean isometries. One way to combine these symmetries is through a hierarchy. However, this approach does not adequately model the interplay between the two levels of symmetry transformations. We show how to model this interplay using ideas from group theory, identify the equivariant linear maps, and introduce equivariant padding that respects these symmetries. Deep networks that use these maps as their building blocks generalize gauge equivariant CNNs on pixelized spheres. These deep networks achieve state-of-the-art results on semantic segmentation for climate data and omnidirectional image processing. Code is available at https://git.io/JGiZA.

Introduction: The Challenge of Spherical Geometry

Spherical data is ubiquitous in scientific and consumer applications, ranging from global climate modeling and astronomy to virtual reality and omnidirectional video. Standard Convolutional Neural Networks (CNNs) are designed for flat, rectangular grids. To apply them to spherical data, researchers often "pixelize" the sphere by projecting it onto a Platonic solid—such as a cube or icosahedron—and tiling each face with a regular grid.

Pixelization of Platonic solids at increasing resolutions, providing a discrete approximation of spherical geometry.

A fundamental challenge in this domain is equivariance: if an input sphere is rotated, the network’s internal representations and final outputs should transform in a mathematically consistent manner. While previous "Gauge Equivariant CNNs" treated each face as an independent local chart, they often ignored the crucial geometric reality that global rotations of the solid dictate how local grids on each face must rotate and shift. This paper introduces a framework to model this interplay explicitly.

Methodology: Modeling Symmetry Interplay

The core contribution of this work is a hierarchical approach to symmetry that moves beyond simple face-shuffling.

Group-Theoretic Interplay and the "Rolling" Effect

The authors utilize the concept of block systems from permutation group theory. The global symmetry of the Platonic solid (the "outer" group) and the local isometries of the face grids (the "inner" group) are combined into a semi-direct product. This ensures that when the solid rotates, the pixels on its faces are transformed in a way that is physically and geometrically consistent.

The commutative diagram illustrating the interplay between global symmetry transformations ($L_G$) and local pixel-level symmetries ($\rho^*$) on a pixelized cube.

A key insight is the "rolling" effect: a 90° rotation of a cube does not just move the top face to the side; it also "rolls" the grid orientation. The proposed mathematical framework captures this interplay, whereas previous models treated face rotations as independent of face permutations. By constraining the model to respect these symmetries, the authors provide a powerful "inductive bias." The architecture's geometry assumes that features, such as a storm system, look the same regardless of their location on the sphere.

Equivariant Linear Maps and Padding

The authors identify the exact parameter-sharing patterns (kernels) required for these complex symmetries. They provide an "orbit-finding" algorithm that automatically generates these weight-sharing matrices for any chosen Platonic solid and tiling type. Furthermore, to allow information to flow across the edges of the Platonic solid without breaking symmetry, they developed a novel equivariant padding procedure. This ensures that the "neighbors" of a pixel on one face are correctly identified on an adjacent face, even if the orientation of the grid changes across the edge.

For tasks like 360-degree photography where "up" and "down" matter, they show how to restrict the symmetry group to only include rotations around the vertical axis. Additionally, to balance expressivity with computational efficiency, they found that applying the expensive global symmetry operations to only a fraction of the network's channels (e.g., 50%) often yielded the best performance.

Experimental Results

The model was evaluated on three distinct benchmarks, demonstrating the power of symmetry as a strong inductive bias.

  • HAPPI20 Climate Data: In high-resolution climate simulations used to detect extreme weather events, the model achieved a Mean Average Precision (mAP) of 95.20%, a significant improvement over the previous state-of-the-art of approximately 80.6%.
  • Stanford 2D3DS: For semantic segmentation of 360-degree indoor images, the "cube + orientation" model reached 62.5% Mean Accuracy, outperforming prior spherical CNN methods (54-58%).
  • Spherical MNIST: The model achieved 99.42% accuracy, matching the performance of specialized spherical models on rotated digit recognition.

Discussion and Future Work

While the framework provides a robust solution for many Platonic solids, some constraints remain. The current treatment of triangular tilings (e.g., on an icosahedron) requires conversion to hexagonal grids to maintain translational symmetry. Additionally, the dodecahedron is excluded because its pentagonal faces cannot be tiled with a regular grid possessing translational symmetry.

Future research may explore "systems of blocks" as a general way to formalize hierarchies of symmetries in other domains. Additionally, while efficient "Pool-Broadcast" operations were implemented, managing these symmetries at extremely high resolutions remains a computational challenge for scaling to very large models.

Summary updated

Abstract

Pixelizations of Platonic solids such as the cube and icosahedron have been widely used to represent spherical data, from climate records to Cosmic Microwave Background maps. Platonic solids have well-known global symmetries. Once we pixelize each face of the solid, each face also possesses its own local symmetries in the form of Euclidean isometries. One way to combine these symmetries is through a hierarchy. However, this approach does not adequately model the interplay between the two levels of symmetry transformations. We show how to model this interplay using ideas from group theory, identify the equivariant linear maps, and introduce equivariant padding that respects these symmetries. Deep networks that use these maps as their building blocks generalize gauge equivariant CNNs on pixelized spheres. These deep networks achieve state-of-the-art results on semantic segmentation for climate data and omnidirectional image processing. Code is available at https://git.io/JGiZA.
BibTeX
@inproceedings{shakerinava2021equivariant,
  author = {Mehran Shakerinava and Siamak Ravanbakhsh},
  title = {{Equivariant Networks for Pixelized Spheres}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=M_TiE6CslH0R},
}

Loading related content...

Figure