Accurate Target Annotation in 3D from Multimodal Streams

Oswald Lanz, Alessio Brutti, Alessio Xompero, Xinyuan Qian, Maurizio Omologo, Andrea CavallaroICASSP 2019 · OpenReview

Abstract

Accurate annotation is fundamental to quantify the performance of multi-sensor and multi-modal object detectors and trackers. However, invasive or expensive instrumentation is needed to automatically generate these annotations. To mitigate this problem, we present a multi-modal approach that leverages annotations from reference streams (e.g. individual camera views) and measurements from unannotated additional streams (e.g. audio) to infer 3D trajectories through an optimization. The core of our approach is a multi-modal extension of Bundle Adjustment with a cross-modal correspondence detection that selectively uses measurements in the optimization. We apply the proposed approach to fully annotate a new multi-modal and multi-view dataset for multi-speaker 3D tracking.
BibTeX
@inproceedings{lanz2019accurate,
  author = {Oswald Lanz and Alessio Brutti and Alessio Xompero and Xinyuan Qian and Maurizio Omologo and Andrea Cavallaro},
  title = {{Accurate Target Annotation in 3D from Multimodal Streams}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  url = {https://openreview.net/forum?id=vUAKlYGOHgz},
}

Loading related content...

Figure