Standard Convolutional Neural Networks (CNNs) possess a natural "superpower" known as translation equivariance. If an object shifts within an image, the network's internal activations shift accordingly. However, as noted by Yuexi Du and Jiazhen Zhang in GMR-Conv, this superpower does not extend to rotations or reflections. A network trained to recognize an upright cat may fail to identify the same cat if it is rotated 90 degrees, unless it has been explicitly trained on vast amounts of augmented data.
To solve this, researchers are moving away from learning arbitrary pixel weights and toward kernels defined by the physics of continuous space: Partial Differential Operators (PDOs).
The Core Idea: Kernels as Differential Operators
In a traditional CNN, a $3 \times 3$ kernel is just nine independent numbers. In contrast, the framework proposed by Zhengyang Shen and Zhouchen Lin in PDO-eConvs models convolutional filters as linear combinations of PDOs.
Why use differential operators? Because the way derivatives (like $\frac{\partial}{\partial x}$ or $\frac{\partial^2}{\partial x \partial y}$) change under rotation is mathematically well-understood. By restricting a kernel to be a combination of these operators, we can "steer" the filter to any orientation. This allows the network to maintain equivariance—ensuring that if the input rotates, the features rotate in a predictable, mathematically guaranteed way.
The Discretization Barrier
The primary challenge in this approach is that while differential operators are defined in continuous space, images and 3D models live on discrete grids (pixels and voxels). When you map a perfectly circular, continuous operator onto a square grid, you introduce discretization errors. These errors act as "noise" that breaks the theoretical symmetry of the model.
Research in this field identifies two primary ways to bridge this gap:
- Finite Differences: In PDO-eConvs, the authors use numerical discretization schemes like finite differences to map operators to pixels. This is highly parameter-efficient but can be sensitive to the grid spacing.
- Gaussian Smoothing: To achieve smoother, more robust symmetry, PDO-s3DCNNs and GMR-Conv employ Gaussian derivatives. By convolving the differential operators with a Gaussian kernel, the model "blurs" the discrete boundaries of the grid.
Why Gaussian Discretization Wins
The shift from finite differences to Gaussian-based methods represents a significant leap in performance for arbitrary rotations. As Zhengyang Shen demonstrates in PDO-s3DCNNs, finite difference methods are often limited to 90-degree "cubic" rotations because they rely strictly on the grid's axes.
Gaussian discretization, however, preserves equivariance for arbitrary angles. The inherent smoothing of the Gaussian kernel mitigates the aliasing effects that occur when a continuous function is sampled onto a square or cubic grid. Yuexi Du further refines this in GMR-Conv by using Gaussian Mixture Rings. By defining kernels as concentric rings weighted by Gaussians, the model ensures radial symmetry, making the kernel look "round" even when it is made of square pixels.
Efficiency and Performance
The practical upshot of using PDO-based kernels is a dramatic reduction in model size. Because the network only needs to learn a few coefficients for a set of fixed differential operators—rather than learning every pixel in every filter—these models can achieve state-of-the-art results with a fraction of the parameters.
For example, PDO-eConvs reached top-tier performance on rotated MNIST datasets while being significantly more compact than traditional models. Similarly, in 3D tasks like shape retrieval and segmentation, PDO-s3DCNNs outperformed existing steerable CNNs by providing a more mathematically rigorous way to handle the complexities of 3D rotation subgroups.