Anisotropic Convolutions for Geometric Deep Learning

42 concepts26 papers1 proposal

In standard image processing, convolutional neural networks (CNNs) owe their immense success to a simple geometric property: the pixel grid has a natural, fixed orientation. When a $3 \times 3$ filter slides across an image, the top-left weight always multiplies the top-left neighbor. This allows the network to learn that a vertical edge looks fundamentally different from a horizontal one. The filter is anisotropic—its behavior changes depending on the spatial direction.

When we move from flat grids to non-Euclidean domains—such as irregular graphs, 3D meshes, or spherical surfaces—this natural coordinate system vanishes. A node in a graph or a vertex on a mesh has no inherent concept of "up," "down," "left," or "right." To handle this, standard Graph Neural Networks (GNNs) rely on permutation-invariant aggregation functions (like summing or averaging neighbor features).

However, this invariance comes at a steep cost: it forces the filters to be isotropic (circularly symmetric). An isotropic filter treats all neighbors identically, regardless of where they sit. As a result, standard geometric networks struggle to capture directional features, smooth out sharp edges, and fail to track oriented physical phenomena like wind fronts or ocean currents.

To overcome this limitation, a prominent research direction focuses on introducing anisotropy into geometric deep learning. By breaking local symmetries, these methods allow filters to become direction-aware while preserving the underlying geometric structure of the domain.


Imposing Order via Local Reference Frames

The most direct way to make a geometric filter direction-aware is to establish a local coordinate system at each point, allowing the network to distinguish neighbors based on their relative orientation.

In the context of physical simulations and trajectory forecasting, Miltiadis Kofinas et al. demonstrate this intuition in Roto-translated Local Coordinate Frames For Interacting Dynamical Systems. By transforming object states into independent local reference frames, the network can employ anisotropic filters where weights are explicitly conditioned on the relative angular and linear positions of neighbors (e.g., distinguishing whether another object is ahead or behind).

For continuous surfaces and 3D meshes, Pim De Haan et al. introduce Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs. This framework defines anisotropic kernels within a local tangent plane at each vertex, allowing the network to differentiate neighbors by their relative angles and significantly improving performance on orientation-dependent tasks.

Isotropic Aggregation (Standard GNN)       Anisotropic Aggregation (Direction-Aware)
         ( Neighbor )                                     ( Neighbor: North )
              │                                                    │
(Neighbor) ───┼─── (Neighbor)                    (Neighbor: West) ───┼─── (Neighbor: East)
              │                                                    │
         ( Neighbor )                                     ( Neighbor: South )
   [All neighbors treated equally]                  [Weights depend on relative angle]

Instead of continuous angles, some architectures enforce directionality by establishing a discrete, consistent ordering of local neighbors. Giorgos Bouritsas et al. propose this in Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation, using a fixed spiral serialization of vertices to break permutation invariance and capture fine-grained structural deformations. To handle meshes where a fixed topology cannot be assumed, Zhongpai Gao et al. develop Learning Local Neighboring Structure for Robust 3D Shape Representation, which learns to "soft-permute" local neighbors into a canonical order before applying a shared anisotropic filter.

A key challenge when defining local frames is that choosing an arbitrary starting orientation can introduce mathematical discontinuities. To resolve this, Ya-Wei Eileen Lin and Ron Levie propose Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks. Their framework allows the neural network to dynamically optimize its own internal reference frame based on the input data, ensuring stable, direction-sensitive feature extraction.


Steering Filters with Intrinsic Fields and Vector Flows

When a surface or graph lacks an obvious coordinate system, researchers look to intrinsic geometric properties or mathematical vector fields to "steer" the direction of the filters.

In 3D computer vision, a surface's principal curvatures offer a natural, intrinsic orientation. Davide Boscaini et al. pioneered this concept in Learning shape correspondence with anisotropic convolutional neural networks by aligning convolutional filters with these principal curvature directions. This mechanism allows the model to resolve rotational ambiguity and distinguish between distinct surface orientations (such as vertical versus horizontal structures on a manifold).

For abstract graphs that do not possess a geometric embedding, defining a direction is less straightforward. Dominique Beaini et al. solve this in Directional Graph Networks by using the gradients of Laplacian eigenvectors as a directional vector field. This field acts as a compass across the graph, guiding message passing and allowing the network to mimic the directional kernels of conventional CNNs.

Similarly, for directed graphs where the asymmetry is embedded in the connectivity itself, Federico Monti et al. introduce MotifNet: a motif-based Graph Convolutional Network for directed graphs. By leveraging local sub-graph patterns (motifs) to construct multiple non-symmetric Laplacians, the network successfully isolates and learns from directional information flows.


Directional Spectral and Spherical Operators

Anisotropy can also be formulated globally through the frequency domain or specialized spherical functions, bypassing the need for explicit local pixel neighborhoods.

Standard spectral graph convolutions are inherently isotropic because they rely on the standard Laplacian operator. To inject directionality, Mohammad Farazi et al. in Anisotropic Multi-Scale Graph Convolutional Network for Dense Shape Correspondence and Hongyan Liu et al. in An anisotropic Chebyshev descriptor and its optimization for deformable shape correspondence utilize an anisotropic Laplace-Beltrami operator. This operator steers diffusion along specific geometric features, breaking intrinsic symmetries that isotropic methods cannot distinguish.

Taking this further, Qinsong Li et al. develop Shape correspondence using anisotropic Chebyshev spectral CNNs by replacing the standard Laplacian with an operator that incorporates a rotatable thermal conductivity tensor, allowing the network to evaluate features at multiple explicit orientations. In the frequency domain, Gijs Joppe Moens et al. introduce SONIC: Spectral Oriented Neural Invariant Convolutions, parameterizing spectral filters as a continuous superposition of directional modes to capture oriented structures like edges and textures efficiently.

When dealing with spherical data, such as 360-degree omnidirectional images, standard spherical harmonics suffer from the same isotropic limitations. To resolve this:


Capturing Directional Dynamics in the Real World

Why does this mathematical pursuit of anisotropy matter? In many real-world applications, the underlying data is fundamentally driven by directional relationships.

In geometric modeling, Zhixiang Chen demonstrates in Learning Feature Aggregation for Deep 3D Morphable Models that introducing learned compatibility weights into mesh aggregation pipelines creates an anisotropic effect. This allows the network to capture complex, non-linear directional relationships between vertices, which is essential for reconstructing detailed 3D shapes.

For geophysical and meteorological data, physical laws are inherently anisotropic. Haitao Lin shows in Conditional Local Convolution for Spatio-Temporal Meteorological Forecasting that preserving relative orientation through specialized coordinate projections allows graph models to track the directional movement of weather fronts.

Similarly, when simulating planetary fluid dynamics on unstructured grids, Neng Shi et al. develop GNN-Surrogate: A Hierarchical and Adaptive Graph Neural Network for Parameter Space Exploration of Unstructured-Mesh Ocean Simulations. By utilizing edge-conditioned convolutions to incorporate geographic directional priors (such as distinguishing between horizontal currents and vertical depth movements), their model successfully mirrors the anisotropic nature of geophysical fluids, accelerating complex climate simulations by orders of magnitude.

Key Foundational Work

MOTIFNET: A Motif-Based Graph Convolutional Network for Directed Graphs (2018) — contributed 1 concept elements to this direction.
Learning shape correspondence with anisotropic convolutional neural networks (2016) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_anisotropic_spherical_convolution_1bdb81c8,
  title = {anisotropic spherical convolution},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/anisotropic-spherical-convolution/rdr_1bdb81c8ae4c4a688af3908b36333996},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_anisotropic_spherical_convolution_1bdb81c8_v1,
  title = {anisotropic spherical convolution},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/anisotropic-spherical-convolution/rdr_1bdb81c8ae4c4a688af3908b36333996/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake