Abstract
Standard Group-equivariant Convolutional Neural Networks (G-CNNs) maintain spatial relationships across geometric transformations but often rely on simple global average pooling to achieve the final invariant representation required for classification. This paper identifies this transition as a "weak link" that discards significant structural information. To address this, we introduce a novel invariant transition layer utilizing orthogonal moments—specifically Zernike, Pseudo-Zernike, and Orthogonal Fourier-Mellin moments. By capturing high-order statistics of feature maps, our approach provides a mathematically robust and descriptive representation. Experimental results on Rotated MNIST and CIFAR-10 demonstrate that this method significantly improves classification accuracy with negligible increases in parameter count.
Introduction
Convolutional Neural Networks (CNNs) are inherently equivariant to translations; however, they lack inherent robustness to other geometric transformations such as rotations and reflections. While Group-equivariant CNNs (G-CNNs) have been developed to preserve these relationships throughout the network's layers, a fundamental challenge remains: the final classification layer must be invariant to these transformations. A "2" must be recognized as a "2" regardless of its orientation.
Currently, most architectures achieve this invariance through global average pooling (GAP) or max pooling across the transformation group. While effective for reducing dimensionality, these methods act as a bottleneck, discarding the intricate geometric structures captured by the equivariant filters. This paper proposes replacing these simplistic pooling operations with a transition layer based on orthogonal moments, allowing the network to retain and utilize high-order structural information.
Methodology: Invariant Transition using Orthogonal Moments (ORIMs)
The proposed framework, termed Invariant Transition using Orthogonal Moments (ORIMs), integrates continuous orthogonal moments into the G-CNN pipeline. The process follows a structured three-step approach:
1. Equivariant Feature Extraction
The network utilizes a G-CNN backbone to produce feature maps that track transformations across the input. To consolidate the group dimension (the various rotated or reflected versions of a filter), a maximum projection is applied over the transformation axis.
2. Moment Calculation
Instead of standard pooling, the architecture inserts a layer that calculates continuous orthogonal moments on the resulting feature maps. The authors explore three specific types of moments:
- Zernike Moments (ZMs): Descriptors defined on a unit disk, widely used in image analysis for their rotationally invariant properties.
- Pseudo-Zernike Moments (PZMs): An evolution of ZMs that offers a greater number of moment coefficients for the same order, providing a more detailed feature representation and improved noise robustness.
- Orthogonal Fourier-Mellin Moments (OFMMs): Moments that are particularly effective at describing small-scale functions and local features.
3. Achieving Invariance
Mathematically, rotating an image results in a phase shift in its complex moments. By extracting the magnitude of these moments, the network generates a representation that is theoretically guaranteed to be invariant to rotation and reflection. This magnitude-based feature vector is then passed to the final fully connected layers for classification.
Experimental Results
The efficacy of ORIMs was evaluated across several standard benchmarks, focusing on datasets where rotation and reflection are prevalent.
Rotated MNIST
On the Rotated MNIST dataset, where digits are randomly rotated between 0 and 2π, the G-CNN + PZMs model achieved a test error of 1.62%. This represents a substantial improvement over the baseline G-CNN (2.28%) and standard CNNs (5.03%). Notably, it also outperformed H-Net (1.69%), a specialized rotation-equivariant architecture.
CIFAR-10 and CIFAR-10+
On the standard CIFAR-10 dataset, the inclusion of PZMs reduced the error rate from 7.59% (standard G-CNN) to 6.90%. Similar gains were observed on CIFAR-10+, which includes horizontal flips and translations, confirming that the moment-based transition enhances robustness even in more complex natural image datasets.
Parameter and Computational Efficiency
A key advantage of this approach is its efficiency. The transition layer is computationally "cheap" regarding model size; for the MNIST model, the parameter count only increased from 22k to 25k. This suggests that the performance gains are derived from better structural representation rather than simply increasing the capacity of the network.
Discussion and Insights
The research highlights several critical insights into the use of high-order statistics in deep learning:
- Orthogonality and Redundancy: Because the basis functions of these moments are orthogonal, the extracted features are highly uncorrelated. This ensures the network captures a complete and non-redundant aspect of the input geometry.
- The Superiority of PZMs: Experiments consistently showed that Pseudo-Zernike Moments outperformed Zernike Moments. The increased number of coefficients in PZMs allows for a more granular description of the feature map, making the model less sensitive to minor distortions.
- Hyperparameter Tuning: The performance of the layer is sensitive to the moment order ($p_{max}$). The study found that ZMs performed best at order 9, while PZMs and OFMMs reached peak performance at order 5.
Conclusion and Future Work
By integrating orthogonal moments into equivariant architectures, this paper provides a mathematically grounded method for preserving structural information during the transition to invariant representations. While the current implementation uses fixed, hand-crafted kernels, the significant performance boost suggests a promising new direction for G-CNN design. Future research may explore learnable moment-based layers or further optimization of the computational complexity associated with calculating these moments across high-dimensional feature maps.
Summary updated