MLVCNN: Multi-Loop-View Convolutional Neural Network for 3D Shape Retrieval

Jianwen Jiang, Di Bao, Ziqiang Chen, Xibin Zhao, Yue Gao

Introduction

3D shape retrieval and classification are fundamental tasks in computer vision, with critical applications in fields like autonomous driving, robotics, and 3D printing. While view-based methods—which represent 3D objects as a collection of 2D snapshots—have demonstrated superior performance over voxel or point-cloud approaches, they often struggle with the hierarchical organization of these views. Traditional methods frequently treat views as an unordered "bag of features" or a simple linear sequence, failing to capture the intrinsic spatial and structural relationships inherent in how an object is observed from different rotations.

The MLVCNN Framework

To address these limitations, the Multi-Loop-View Convolutional Neural Network (MLVCNN) introduces a hierarchical approach that organizes views into multiple orthogonal loops. This structure allows the model to capture comprehensive geometric information while preserving the local discriminativeness of specific view sequences.

Multi-Loop Projection

Unlike standard methods that might use a single ring of cameras, MLVCNN employs three orthogonal loops aligned with the X, Y, and Z axes. This ensures that all facets of an object—including the top and bottom, which are often overlooked—are captured.

Multi-loop camera setup and resulting view sequences for a 3D car model.

Hierarchical Architecture

The MLVCNN architecture processes information through three distinct stages: View-level, Loop-level, and Shape-level descriptor generation.

The three-stage hierarchical architecture of MLVCNN, including the LoopNorm block.
  1. View-level Descriptor Generation: Individual views from each loop are processed through a standard CNN (such as ResNet18) to extract high-level visual features.
  2. Loop-level Descriptor Generation: This stage introduces the core innovations of the paper:
  3. Loop Normalization (LN): The authors identify that global Batch Normalization can dilute the unique statistical signatures of different rotations. LN normalizes features within each specific loop, preserving the local details that distinguish one perspective from another.
  4. LSTM Integration: Since views within a loop follow a spatial sequence, Long Short-Term Memory (LSTM) networks are used to aggregate these features, effectively "stitching" them into a coherent loop descriptor.
  5. Shape-level Descriptor Generation: Finally, descriptors from all loops are concatenated and passed through a fully connected layer to produce a single, robust embedding for the entire 3D shape.

Experimental Results

The model was rigorously evaluated on the ModelNet40 dataset, a benchmark containing over 12,000 3D shapes. MLVCNN achieved state-of-the-art results in both retrieval and classification tasks.

  • Retrieval: The model reached a mean Average Precision (mAP) of 91.15%, which increased to 92.84% when utilizing Center Loss. This represents a significant 4.84% improvement over previous leading methods.
  • Classification: MLVCNN achieved an accuracy of 94.16%, outperforming other prominent architectures like GVCNN and PointNet++.

Ablation studies confirmed the importance of the multi-loop approach, showing that three orthogonal loops provide a substantial performance boost (~5% mAP) compared to a single-loop setup. Furthermore, the inclusion of Loop Normalization consistently improved results by approximately 2%.

Conclusion

MLVCNN advances 3D shape analysis by moving beyond flat view sets to a structured, hierarchical representation. By combining orthogonal multi-loop projection with specialized normalization and sequential processing, the framework captures the complex geometry of 3D objects more effectively than previous view-based models. While the computational demand of processing multiple loops remains a consideration, the significant gains in retrieval and classification accuracy mark a new benchmark for the field.

Summary updated

Abstract

3D shape retrieval has attracted much attention and become a hot topic in computer vision field recently.With the development of deep learning, 3D shape retrieval has also made great progress and many view-based methods have been introduced in recent years. However, how to represent 3D shapes better is still a challenging problem. At the same time, the intrinsic hierarchical associations among views still have not been well utilized. In order to tackle these problems, in this paper, we propose a multi-loop-view convolutional neural network (MLVCNN) framework for 3D shape retrieval. In this method, multiple groups of views are extracted from different loop directions first. Given these multiple loop views, the proposed MLVCNN framework introduces a hierarchical view-loop-shape architecture, i.e., the view level, the loop level, and the shape level, to conduct 3D shape representation from different scales. In the view-level, a convolutional neural network is first trained to extract view features. Then, the proposed Loop Normalization and LSTM are utilized for each loop of view to generate the loop-level features, which considering the intrinsic associations of the different views in the same loop. Finally, all the loop-level descriptors are combined into a shape-level descriptor for 3D shape representation, which is used for 3D shape retrieval. Our proposed method has been evaluated on the public 3D shape benchmark, i.e., ModelNet40. Experiments and comparisons with the state-of-the-art methods show that the proposed MLVCNN method can achieve significant performance improvement on 3D shape retrieval tasks. Our MLVCNN outperforms the state-of-the-art methods by the mAP of 4.84% in 3D shape retrieval task. We have also evaluated the performance of the proposed method on the 3D shape classification task where MLVCNN also achieves superior performance compared with recent methods.
BibTeX
@inproceedings{jiang2019mlvcnn,
  author = {Jianwen Jiang and Di Bao and Ziqiang Chen and Xibin Zhao and Yue Gao},
  title = {{MLVCNN: Multi-Loop-View Convolutional Neural Network for 3D Shape Retrieval}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33018513},
}
Figure