RF-based Multi-view Pose Machine for Multi-Person 3D Pose Estimation
Lists
Only you see this
Abstract
In this paper, we present RF-based Multi-view Pose machine (RF-MvP) for multi-person 3D pose estimation using RF signals. Specifically, we first develop a lightweight anchor-free detector module to locate and crop regions of interest from horizontal and vertical RF signals. Afterward, we propose a Multi-view Fusion Network to unproject the RF signals from the horizontal and vertical millimeter-wave radars into a unified latent space, and then calculate the correlation for weighted fusion. Finally, a Spatio-Temporal Attention Network is designed to reconstruct the multi-person 3D skeleton sequences, in which the spatial attention module is proposed to recover invisible body parts using non-local correlations among joints and the temporal attention module refines the 3D pose sequences using temporal coherency learned from frame queries. We evaluate the performance of the proposed RF-MvP and state-of-the-art methods on a large-scale dataset with multi-person 3D pose labels and corresponding radar signals. The experimental results show that RF-MvP outperforms all of the baseline methods, which locates multi-person 3D key points with an average error of 73mm and generalizes well in new data such as occlusion, low illumination.
BibTeX
@inproceedings{xie2023rf,
author = {Chunyang Xie and Dongheng Zhang and Zhi Wu and Cong Yu and Yang Hu and Qibin Sun and Yan Chen},
title = {{RF-based Multi-view Pose Machine for Multi-Person 3D Pose Estimation}},
booktitle = {ICME},
year = {2023},
url = {https://openreview.net/forum?id=ia7f4XsrDb},
}Loading related content...