How Powerful Are Randomly Initialized Pointcloud Set Functions?

Aditya Sanghi, Pradeep Kumar JayaramanarXiv 2020 · arXiv

TL;DR

This paper demonstrates that untrained, randomly initialized neural networks for point clouds produce surprisingly powerful embeddings. These random representations are often linearly separable and capture essential global shape features, achieving classification accuracies on benchmarks like ModelNet40 that come remarkably close to fully trained state-of-the-art models.

What Problem Does This Paper Address?

In the world of 3D computer vision, point clouds are a standard way to represent data. However, processing them requires specialized "set functions"—neural networks designed to be invariant to the order of points. Typically, these networks require extensive training on massive datasets to learn meaningful features.

The authors ask a fundamental question: How much of a model's success is due to its learned weights versus its inherent architectural structure? By treating untrained networks as random nonlinear projection functions, they explore whether the mere structure of these networks is enough to create useful data representations for tasks like classification and reconstruction.

Methods

The researchers took several well-known point cloud architectures—including PointNet, DeepSets, and a simplified linear model called LinSet—and initialized their weights randomly using standard methods (like Glorot initialization). Crucially, they did not train these encoders.

Instead, they used these random networks to generate 1024-dimensional embeddings for input point clouds. They then evaluated these "frozen" embeddings by training two types of simple downstream classifiers:

  1. LINCLF: A basic 1-layer linear classifier.
  2. NONLINCLF: A 3-layer non-linear classifier.

The study also investigated how specific architectural choices—such as the number of layers and different types of normalization (Batchnorm, InstanceNorm, LayerNorm)—affected the quality of these random embeddings.

Datasets

The authors evaluated their approach on several standard benchmarks:

  • MNIST-PC: A point cloud version of the classic MNIST digit dataset, created by sampling points from the original images.
  • ModelNet40 & ModelNet10: The gold standard for 3D object classification, containing thousands of CAD models across 40 (or 10) categories.
  • ShapeNet (CHAIR category): Used specifically for qualitative reconstruction experiments to see if a decoder could "recover" a shape from a random embedding.

Key Insights and Ideas

  • Architecture is a Prior: Even without training, the architectural design of a network acts as a powerful "prior." PointNet, in particular, produced embeddings that were highly effective for classification right out of the box.
  • Normalization is Non-Negotiable: Normalization layers (like InstanceNorm) are critical for making random embeddings linearly separable. Without them, accuracy for linear classifiers plummeted.
  • The "More Layers" Trap: In unnormalized random networks, adding more layers actually decreased performance, suggesting that deep random projections without normalization eventually scramble the data into an unusable state.
  • Global Structure Preservation: Strikingly, random embeddings capture the global "gist" of a shape. The authors proved this by training a decoder to reconstruct 3D chairs from random embeddings; the results were surprisingly recognizable.
Figure 1: Autoencoder reconstructions for the CHAIR category from ShapeNet. Column 1: Ground truth, Column 2: Fully trained PointNet autoencoder, Column 3: Random encoder + trained decoder.
  • Alignment Matters: Random embeddings are highly sensitive to the orientation of the point cloud. When the dataset was randomly rotated, performance dropped significantly, indicating that these networks rely heavily on the canonical alignment found in standard datasets.

Key Results

The quantitative results were unexpectedly strong. As shown in the table below, an untrained PointNet combined with a simple non-linear classifier achieved 85.24% accuracy on ModelNet40. For context, a fully trained PointNet achieves about 89.20%.

Table 1: Comparison of classification accuracy between random networks and fully trained models.

Qualitatively, t-SNE visualizations revealed that these random embeddings naturally cluster by object class, even though the network has never "seen" a label or adjusted a single weight to minimize error.

Figure 2: t-SNE visualization of random embeddings.

Limitations and Future Work

The primary limitation identified is rotation sensitivity. Because the random projections are not rotation-invariant, the model's performance relies on the dataset being pre-aligned.

The authors suggest that future work should explore:

  • Rotation-invariant architectures: Testing if random versions of these models can overcome the alignment requirement.
  • Theoretical Foundations: Developing a deeper mathematical understanding (perhaps extending Cover’s Theorem) of why these random set functions are so effective.
  • Training Efficiency: Using random networks as strong, "cheap" baselines or as starting points to help trained encoders learn even better representations.

Summary updated

Abstract

We study random embeddings produced by untrained neural set functions, and show that they are powerful representations which well capture the input features for downstream tasks such as classification, and are often linearly separable. We obtain surprising results that show that random set functions can often obtain close to or even better accuracy than fully trained models. We investigate factors that affect the representative power of such embeddings quantitatively and qualitatively.
BibTeX
@misc{sanghi2020how,
  author = {Aditya Sanghi and Pradeep Kumar Jayaraman},
  title = {{How Powerful Are Randomly Initialized Pointcloud Set Functions?}},
  year = {2020},
  eprint = {2003.05410},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2003.05410},
}
Figure