Abstract
Recognition of 3D shapes is a fundamental task in computer vision. In recent years, view-based deep learning has emerged as an effective approach. However, most existing methods treat the views of an object as an unordered set, ignoring dynamic relations like sequential semantic dependencies. This paper proposes modeling views as a sequence to exploit long-term dependencies using a sequence-aware view aggregation module based on a bi-directional Long Short-Term Memory (Bi-LSTM) network. This module captures bi-directional dependencies and is robust to circular shifts. Evaluated on ModelNet and ShapeNet datasets, the method demonstrates significant performance gains in classification and retrieval.
Introduction
3D object recognition is a cornerstone of modern computer vision, essential for applications like autonomous driving and robotics. View-based recognition, which represents 3D objects through a collection of 2D images, has proven highly effective. Despite its success, a major limitation persists: most frameworks (e.g., MVCNN) treat these views as an unordered set. By applying simple max or average pooling, these methods discard the spatial and sequential context inherent in how views are captured. This paper addresses this gap by treating views as a sequence, allowing the model to learn the "dynamic relations" between different angles of an object.
Methodology
The proposed framework utilizes a three-stage pipeline to process 3D shapes.
- Multi-View Representation: The system renders 12 views of a 3D model by placing virtual cameras at 30-degree intervals around the object's principal axis.
- Feature Extraction: Each view is processed by a shared VGG-11 Convolutional Neural Network (CNN) with Batch Normalization to extract high-level feature vectors.
- Sequence-Aware View Aggregation: This is the core innovation. Instead of pooling, the feature vectors are fed into a Bi-directional LSTM (Bi-LSTM). The Bi-LSTM processes the sequence in both forward and backward directions, enabling the network to link distant but semantically related views (e.g., the front and back of a car).
Key Insights: Symmetry and Sequentiality
The primary insight of this work is that geometric properties like symmetry manifest as long-range correlations in a view sequence. A Bi-LSTM is uniquely suited to "bridge" these dependencies.
Furthermore, the architecture is naturally robust to "circular shifts." Regardless of which angle the camera starts at, the Bi-LSTM captures the full global context. This makes the representation invariant to the initial camera orientation, a significant advantage over standard unidirectional LSTMs.
Experimental Results
The model was benchmarked against standard datasets:
- ModelNet40/10: The method achieved 94.65% accuracy on ModelNet40 and 95.26% on ModelNet10, outperforming MVCNN (90.1%) and GVCNN (93.1%).
- ShapeNetCore55: In large-scale retrieval tasks, the method achieved a micro-mAP of 0.778, showing superior performance in macro-metrics compared to previous state-of-the-art models like RotationNet.
Ablation studies confirmed that the Bi-LSTM is critical; replacing it with standard pooling or a unidirectional LSTM resulted in a performance drop of 1-2%.
Conclusion
By treating 3D views as a sequence rather than a set, this research demonstrates that long-term dependencies are vital for accurate shape recognition. The Bi-LSTM aggregation module provides a robust and efficient way to capture these relations. Future work may explore attention mechanisms to further refine view prioritization and handle more arbitrary camera viewpoints.
Summary updated