MMJN: Multi-Modal Joint Networks for 3D Shape Recognition
Lists
Only you see this
Abstract
3D shape recognition has attracted wide research attention in the field of multimedia and computer vision. With the recent advance of deep learning, various deep models with different representations have achieved the state-of-the-art performances. Among them, many modalities are proposed to represent 3D model, such as point cloud, multi-view, and PANORAMA-view. Based on these representations, many corresponding deep models have shown significant performances on 3D shape recognition. However, few work to considers utilizing the fusion information of multi-modal for 3D shape recognition. Since these different modalities represent the same 3D model, they should guide each other to get a better feature representation. In this paper, we propose a novel multi-modal joint network (MMJN) for 3D shape recognition, which can consider the correlation between two different modalities to extract the robust feature vector. More specifically, we propose a novel correlation loss which can utilize the correlation between different features extracted by different modality networks to increase the robustness of the feature representation. Finally, we utilize the late fusion method to fuse the multi-modal information for 3D model representation and recognition. Here, we define the weight of different modalities features based on the statistic method and utilize the advantages of different modalities to generate more robust feature. We evaluated the proposed method on the ModelNet40 dataset for 3D shape classification and retrieval tasks. Experimental results and comparisons with the state-of-the-art methods demonstrate the superiority of our approach.
BibTeX
@inproceedings{nie2019mmjn,
author = {Weizhi Nie and Qi Liang and An-An Liu and Zhendong Mao and Yangyang Li},
title = {{MMJN: Multi-Modal Joint Networks for 3D Shape Recognition}},
booktitle = {ACM Multimedia},
year = {2019},
doi = {10.1145/3343031.3351009},
}Loading related content...