Building Kinematic and Dynamic Models of Articulated Objects with Multi-Modal Interactive Perception

Roberto Martín-Martín, Oliver Brock

Abstract

Interactive perception methods exploit the correlation between robot actions and changes in the sensor signals to extract task-relevant information from the sensor stream. These interactions can have effects in one or multiple sensor modalities. In this work we propose an interactive perception system to build kinematic and dynamic models of articulated objects from multi-modal sensor streams. Our system leverages two properties of the multi-modal signals: 1) consequences of actions are frequently best perceived in the combined multi-modal sensor-action stream, and 2) properties perceived in one modality may depend on properties perceived in other modalities. To fully exploit these properties we propose a perceptual architecture based on two main components: a set of (possibly recursive) estimation processes and an interconnection mechanism between them. The processes extract patterns of information from the multi-modal sensor-action stream by enriching it with prior knowledge; the interconnection mechanism leverages their inter-dependencies by using estimations of one process as measurements, priors or forward models of other processes. We show experimentally that our system integrates and balances between different modalities according to their uncertainty in order to overcome limitations of uni-modal perceptual systems.
BibTeX
@inproceedings{martinmartin2017building,
  author = {Roberto Martín-Martín and Oliver Brock},
  title = {{Building Kinematic and Dynamic Models of Articulated Objects with Multi-Modal Interactive Perception}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2017},
}

Loading related content...

Figure