PointListNet: Deep Learning on 3D Point Lists

Hehe Fan, Linchao Zhu, Yi Yang, Mohan S. Kankanhalli

Abstract

PointListNet is a novel deep learning architecture designed specifically for "3D point lists"—data structures like proteins that possess both a regular 1D sequence and irregular 3D spatial coordinates. By replacing standard Transformer self-attention with a non-parametric distance-based mechanism and explicitly modeling relative 1D/3D displacements, the model achieves state-of-the-art performance on protein and enzyme classification tasks.

Introduction: The Challenge of Hybrid Data

Deep learning has mastered two distinct data types: regular 1D lists (like text in NLP) and irregular 3D sets (like point clouds in computer vision). However, biological data like proteins and non-coding RNAs fall into a "hybrid" category.

A protein is a chain of amino acids with a specific 1D sequential order, but it also folds into a complex 3D geometric shape. Existing models often compromise: they either treat the protein as a simple string of letters (ignoring 3D structure) or as an unordered cloud of points (ignoring the vital sequence backbone).

Comparison of data structures: (a) text, (b) image, (c) 3D point cloud, and (d) protein as a 3D point list.

The authors argue that to truly understand these "micro-world" structures, we need a model that treats them as 3D point lists, capturing both the sequence dependency and the spatial geometry simultaneously.

Methodology: Adapting Transformers for the Micro-World

PointListNet adapts the Transformer architecture to handle this hybrid data through two primary innovations:

1. Non-parametric Distance-based Attention

Standard Transformers use "self-attention," which learns to correlate elements based on their features. The authors found that in the micro-world, physical distance is often more important than feature type. They replaced learned attention with a fixed, non-parametric function where attention weight is determined by both 1D sequence distance and 3D Euclidean distance. This is more computationally efficient and focuses the model on local structural interactions.

2. 1D-3D-Displacement-Aware Values

In a vanilla Transformer, "values" are simple linear projections of input features. PointListNet instead integrates relative relations. It calculates values based on the 1D order displacement (how many steps apart in the chain) and the 3D Euclidean displacement (the vector between points). To ensure the model is robust to orientation, they use a rotationally invariant encoding for these 3D displacements.

The network uses a hierarchical (pyramid) architecture, downsampling the points as the network gets deeper to extract global features for classification.

Experimental Results

The authors evaluated PointListNet on two major biological recognition tasks:

  • Protein Fold Classification: Using the SCOPe 1.75 dataset (16,000+ proteins).
  • Enzyme Reaction Classification: A dataset of nearly 30,000 proteins categorized by chemical reaction types.

PointListNet set new benchmarks across the board, significantly outperforming both general 3D point cloud models and specialized protein modeling networks:

  • Protein Fold Classification (Family level): Achieved 99.5% accuracy, outperforming PointNet++ (93.8%) and the specialized IEConv (98.9%).
  • Enzyme Reaction Classification: Achieved 88.0% accuracy, a significant jump over Point Transformer (81.3%) and GearNet (79.4%).

Ablation Study: When the 1D sequence info was removed, enzyme classification accuracy dropped from 88.0% to 84.5%. When 3D info was removed, it plummeted to 70.0%, proving the necessity of the hybrid "3D point list" approach.

Key Insights and Discussion

  • The "Micro-World" Priority: Unlike "macro" objects (like chairs) where semantic parts define the object, protein behavior is heavily dictated by the physical proximity of amino acids. This justifies moving away from feature-heavy self-attention toward distance-based attention.
  • Relative > Absolute: Vanilla Transformers rely on absolute positional embeddings. PointListNet shows that for molecular structures, relative displacements are much more informative and robust.
  • Efficiency Gains: By using a non-parametric attention function, the model reduces the number of trainable parameters and speeds up inference compared to standard self-attention.

Limitations and Future Work

While highly effective, capturing 1D sequence dependencies still requires a large group of independent parameters ($2N-1$ groups). The authors manage this with local modeling thresholds, but it remains a consideration for very long sequences. Future work could explore rotationally equivariant architectures, which might better preserve the directional relationships between amino acids, and explore other "3D point list" applications such as trajectory data.

Summary updated

Abstract

Deep neural networks on regular 1D lists (e.g., natural languages) and irregular 3D sets (e.g., point clouds) have made tremendous achievements. The key to natural language processing is to model words and their regular order dependency in texts. For point cloud understanding, the challenge is to understand the geometry via irregular point coordinates, in which point-feeding orders do not matter. However, there are a few kinds of data that exhibit both regular 1 D list and irregular 3D set structures, such as proteins and non-coding RNAs. In this paper, we refer to them as 3D point lists and propose a Transformer-style PointListNet to model them. First, PointListNet employs non-parametric distance-based attention because we find sometimes it is the distance, instead of the feature or type, that mainly determines how much two points, e.g., amino acids, are correlated in the micro world. Second, different from the vanilla Transformer that directly performs a simple linear transformation on inputs to generate values and does not explicitly model relative relations, our PointListNet integrates the 1D order and 3D Euclidean displacements into values. We conduct experiments on protein fold classification and enzyme reaction classification. Experimental results show the effectiveness of the proposed PointListNet.
BibTeX
@inproceedings{fan2023pointlistnet,
  author = {Hehe Fan and Linchao Zhu and Yi Yang and Mohan S. Kankanhalli},
  title = {{PointListNet: Deep Learning on 3D Point Lists}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01697},
}
Figure