Adaptive Spiral Layers for Efficient 3D Representation Learning on Meshes

Francesca Babiloni, Matteo Maggioni, Thomas Tanay, Jiankang Deng, Ales Leonardis, Stefanos Zafeiriou

Introduction

Deep learning has achieved remarkable success in processing 2D grid-based data, yet 3D meshes—the standard representation in CGI, CAD, and medical imaging—present unique challenges. Unlike regular grids, meshes are non-Euclidean graphs where vertices exhibit irregular connectivity. Previous state-of-the-art approaches, such as SpiralNet++, attempted to bridge this gap by unrolling local neighborhoods into fixed spiral sequences. However, these methods rely on rigid, static operators that fail to account for local geometric variations or long-range dependencies, often leading to high computational costs and limited expressivity.

This paper introduces the Adaptive Spiral Layer, a novel intrinsic operator designed to break the rigidity of standard spiral convolutions. By integrating tensor factorization with dynamic gating and efficient pooling, the proposed model achieves state-of-the-art performance with significantly reduced computational overhead.

Methodology

The core innovation lies in a vertex-adaptive operator that adjusts its focus for every individual vertex. This is achieved through three primary technical contributions:

1. Tensor Factorization (CP Decomposition)

To address the scalability issues of dense weight tensors, the authors factorize the convolution weights into a sequence of lighter linear layers. This separation of "feature" processing from "spatial" (vertex) processing drastically reduces the parameter count and memory footprint without sacrificing representational power.

2. Dynamic Gating and Vertex-Adaptivity

The model implements a vertex-wise gating mechanism. This allows the network to dynamically weight the importance of specific vertices based on their local features and the global mesh structure. This adaptivity enables the model to distinguish between different anatomical regions—such as the fine details of a nose versus the relatively flat surface of a forehead—and process them accordingly.

3. Efficient Global Reasoning via Summed-Area Tables (SAT)

To capture long-range dependencies without the $O(N^2)$ complexity of standard attention mechanisms, the authors adapt Summed-Area Tables (SAT) to 1D spiral trajectories. This technique allows the model to pool information over variable-length spirals in constant time, providing a dynamic receptive field that can scale from immediate neighbors to over 70 vertices as needed.

The architecture of the Adaptive Spiral Layer includes a gated spiral depthwise convolution and a dynamic pooling mechanism, implemented efficiently via tensor operations.

Experimental Results

The proposed architecture was evaluated on several rigorous 3D benchmarks, including FAUST (shape correspondence), COMA (facial expression reconstruction), and SCAPE (generalization across triangulations).

Performance and Efficiency

On the FAUST dataset, the model achieved a remarkable 99.88% exact correspondence accuracy. Crucially, it reached this level of precision using 15x fewer GFlops and 84% fewer parameters than the previous state-of-the-art, SpiralNet++. In 3D reconstruction tasks on the COMA dataset, the method reduced mean error by up to 20.5% compared to baseline methods.

Quantitative and qualitative comparison: The proposed method (red) converges significantly faster and more accurately than SpiralConv variants while maintaining a lower computational footprint (left). Visual error maps (right) demonstrate superior surface reconstruction compared to the baseline.

Visual Adaptivity

Analysis of the learned kernels through t-SNE visualizations revealed that the model naturally clusters kernels according to vertex types. This confirms that the network successfully learns to specialize its operations for different anatomical regions, such as the eyes, chin, or limbs, validating the "vertex-adaptive" design philosophy.

Conclusion and Future Work

The Adaptive Spiral Layer represents a significant step forward in geometric deep learning, offering a highly efficient and flexible alternative to static mesh operators. By leveraging tensor factorization and dynamic pooling, the model achieves superior accuracy with a fraction of the computational cost.

Future research directions include optimizing the initial spiral extraction process for more heterogeneous mesh structures and extending the adaptive logic to other 3D representations, such as point clouds or irregularly sampled 2D signals.

Summary updated

Abstract

The success of deep learning models on structured data has generated significant interest in extending their application to non-Euclidean domains. In this work, we introduce a novel intrinsic operator suitable for representation learning on 3D meshes. Our operator is specifically tailored to adapt its behavior to the irregular structure of the underlying graph and effectively utilize its long-range dependencies, while at the same time ensuring computational efficiency and ease of optimization. In particular, inspired by the framework of Spiral Convolution, which extracts and transforms the vertices in the 3D mesh following a local spiral ordering, we propose a general operator that dynamically adjusts the length of the spiral trajectory and the parameters of the transformation for each processed vertex and mesh. Then, we use polyadic decomposition to factorize its dense weight tensor into a sequence of lighter linear layers that separately process features and vertices information, hence significantly reducing the computational complexity without introducing any stringent inductive biases. Notably, we leverage dynamic gating to achieve spatial adaptivity and induce global reasoning with constant time complexity benefitting from an efficient dynamic pooling mechanism based on Summed-Area-tables. Used as a drop-in replacement on existing architectures for shape correspondence our operator significantly improves the performance-efficiency trade-off, and in 3D shape generation with morphable models achieves state-of-the-art performance with a three-fold reduction in the number of parameters required. Project page: https://github.com/Fb2221/DFC
BibTeX
@inproceedings{babiloni2023adaptive,
  author = {Francesca Babiloni and Matteo Maggioni and Thomas Tanay and Jiankang Deng and Ales Leonardis and Stefanos Zafeiriou},
  title = {{Adaptive Spiral Layers for Efficient 3D Representation Learning on Meshes}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01344},
}
Figure