Abstract
Equivariance to permutations and rigid motions is an important inductive bias for various 3D learning problems. Recently it has been shown that the equivariant Tensor Field Network architecture is universal -- it can approximate any equivariant function. In this paper we suggest a much simpler architecture, prove that it enjoys the same universality guarantees and evaluate its performance on Modelnet40. The code to reproduce our experiments is available at \url{https://github.com/simpleinvariance/UniversalNetwork}
Introduction
Deep learning on 3D point clouds faces two fundamental geometric challenges: permutation invariance and rotation equivariance. A point cloud is essentially a set; therefore, a network's output should remain unchanged regardless of the order in which points are processed. Simultaneously, if a 3D object is rotated, the network’s internal features or final outputs should rotate accordingly (equivariance) or remain fixed (invariance).
While many architectures address these symmetries, few are theoretically "universal"—meaning they are capable of approximating any continuous function that obeys these geometric constraints. Existing universal models, such as Tensor Field Networks (TFN), often rely on complex group theory and spherical harmonics, making them difficult to implement and optimize. This paper proposes a simplified alternative that achieves universality using only standard multilinear algebra.
Methodology: The Semi-Lifted Architecture
The authors propose a "semi-lifted" approach that represents features as tensors of varying orders. In this framework, a tensor of order $k$ represents geometric information: $T_0$ is a scalar, $T_1$ is a 3D vector, and $T_2$ is a $3 \times 3$ matrix.
The U-Net Structure
The network follows a symmetric path designed to process these tensors:
- Ascending Layers: These layers increase the tensor order using the tensor product ($\otimes$). By combining current features with the input point coordinates, the network "lifts" the data into higher-dimensional geometric representations.
- Descending Layers: These layers reduce the tensor order through contractions. Contraction is a generalization of the matrix trace, where indices are summed over to reduce dimensionality while preserving equivariance.
To make the model practical for real-world data, the authors incorporate K-Nearest Neighbors (KNN) to capture local geometric structures and a specialized ReLU variant (Vector Neurons) to introduce non-linearity.
Theoretical Insights
The primary contribution of this work is the proof that simple tensor products and contractions are sufficient to span the space of equivariant polynomials. The expressive power of the network is directly tied to the maximum tensor order $K$. For example, the authors demonstrate that a network requires a tensor order of at least $K=6$ to compute the eigenvalues of a point cloud's covariance matrix—a fundamental geometric descriptor.
Unlike "fully lifted" models that scale exponentially with the number of points $n$, this semi-lifted model scales linearly with $n$. However, it does scale exponentially with the tensor order $K$, as a tensor of order $K$ has $3^K$ components, creating a trade-off between theoretical expressivity and computational cost.
Experimental Evaluation
The model was evaluated on ModelNet40, a standard benchmark for 3D shape classification containing 12,311 CAD models across 40 categories. The authors tested the model under three rotation protocols to verify its equivariant properties:
- z/z: Trained and tested with rotations only around the vertical axis.
- z/SO(3): Trained on z-rotations, tested on arbitrary 3D rotations.
- SO(3)/SO(3): Trained and tested on arbitrary 3D rotations.
Key Results
- Performance vs. Complexity: As predicted by the theory, increasing the maximum tensor order $K$ improved accuracy. On the SO(3)/SO(3) protocol, the model achieved 77.7% accuracy at $K=2$, rising to 80.1% at $K=6$.
- Equivariance Gap: The model showed almost no performance drop when moving from z-axis rotations to full 3D rotations, confirming that the mathematical equivariance was successfully maintained in practice.
- Ablation Impact: The inclusion of non-linearities and local information was critical. Removing these components caused accuracy to plummet from ~80% to 33.8%, demonstrating that while the tensor framework provides the theoretical foundation for universality, standard deep learning techniques provide the necessary empirical performance.
Conclusion
This paper demonstrates that universality in equivariant point-cloud networks can be achieved without the heavy machinery of spherical harmonics. While the current model does not yet reach the state-of-the-art (SOTA) performance of specialized architectures like VN-DGCNN (~90.2%), it provides a simpler, theoretically grounded framework for 3D learning. Future work will focus on optimizing these high-order tensor representations to bridge the gap between theoretical universality and empirical SOTA results.
Summary updated