Abstract
PointListNet is a novel deep learning architecture designed specifically for "3D point lists"—data structures like proteins that possess both a regular 1D sequence and irregular 3D spatial coordinates. By replacing standard Transformer self-attention with a non-parametric distance-based mechanism and explicitly modeling relative 1D/3D displacements, the model achieves state-of-the-art performance on protein and enzyme classification tasks.
Introduction: The Challenge of Hybrid Data
Deep learning has mastered two distinct data types: regular 1D lists (like text in NLP) and irregular 3D sets (like point clouds in computer vision). However, biological data like proteins and non-coding RNAs fall into a "hybrid" category.
A protein is a chain of amino acids with a specific 1D sequential order, but it also folds into a complex 3D geometric shape. Existing models often compromise: they either treat the protein as a simple string of letters (ignoring 3D structure) or as an unordered cloud of points (ignoring the vital sequence backbone).
The authors argue that to truly understand these "micro-world" structures, we need a model that treats them as 3D point lists, capturing both the sequence dependency and the spatial geometry simultaneously.
Methodology: Adapting Transformers for the Micro-World
PointListNet adapts the Transformer architecture to handle this hybrid data through two primary innovations:
1. Non-parametric Distance-based Attention
Standard Transformers use "self-attention," which learns to correlate elements based on their features. The authors found that in the micro-world, physical distance is often more important than feature type. They replaced learned attention with a fixed, non-parametric function where attention weight is determined by both 1D sequence distance and 3D Euclidean distance. This is more computationally efficient and focuses the model on local structural interactions.
2. 1D-3D-Displacement-Aware Values
In a vanilla Transformer, "values" are simple linear projections of input features. PointListNet instead integrates relative relations. It calculates values based on the 1D order displacement (how many steps apart in the chain) and the 3D Euclidean displacement (the vector between points). To ensure the model is robust to orientation, they use a rotationally invariant encoding for these 3D displacements.
The network uses a hierarchical (pyramid) architecture, downsampling the points as the network gets deeper to extract global features for classification.
Experimental Results
The authors evaluated PointListNet on two major biological recognition tasks:
- Protein Fold Classification: Using the SCOPe 1.75 dataset (16,000+ proteins).
- Enzyme Reaction Classification: A dataset of nearly 30,000 proteins categorized by chemical reaction types.
PointListNet set new benchmarks across the board, significantly outperforming both general 3D point cloud models and specialized protein modeling networks:
- Protein Fold Classification (Family level): Achieved 99.5% accuracy, outperforming PointNet++ (93.8%) and the specialized IEConv (98.9%).
- Enzyme Reaction Classification: Achieved 88.0% accuracy, a significant jump over Point Transformer (81.3%) and GearNet (79.4%).
Ablation Study: When the 1D sequence info was removed, enzyme classification accuracy dropped from 88.0% to 84.5%. When 3D info was removed, it plummeted to 70.0%, proving the necessity of the hybrid "3D point list" approach.
Key Insights and Discussion
- The "Micro-World" Priority: Unlike "macro" objects (like chairs) where semantic parts define the object, protein behavior is heavily dictated by the physical proximity of amino acids. This justifies moving away from feature-heavy self-attention toward distance-based attention.
- Relative > Absolute: Vanilla Transformers rely on absolute positional embeddings. PointListNet shows that for molecular structures, relative displacements are much more informative and robust.
- Efficiency Gains: By using a non-parametric attention function, the model reduces the number of trainable parameters and speeds up inference compared to standard self-attention.
Limitations and Future Work
While highly effective, capturing 1D sequence dependencies still requires a large group of independent parameters ($2N-1$ groups). The authors manage this with local modeling thresholds, but it remains a consideration for very long sequences. Future work could explore rotationally equivariant architectures, which might better preserve the directional relationships between amino acids, and explore other "3D point list" applications such as trajectory data.
Summary updated