Towards a Theoretical Framework for Learning Multi-modal Patterns for Embodied Agents

Nicoletta Noceti, Barbara Caputo, Claudio Castellini, Luca Baldassarre, Annalisa Barla, Lorenzo Rosasco, Francesca Odone, Giulio Sandini

Abstract

Multi-modality is a fundamental feature that characterizes biological systems and lets them achieve high robustness in understanding skills while coping with uncertainty. Relatively recent studies showed that multi-modal learning is a potentially effective add-on to artificial systems, allowing the transfer of information from one modality to another. In this paper we propose a general architecture for jointly learning visual and motion patterns: by means of regression theory we model a mapping between the two sensorial modalities improving the performance of artificial perceptive systems. We present promising results on a case study of grasp classification in a controlled setting and discuss future developments.
BibTeX
@inproceedings{noceti2009towards,
  author = {Nicoletta Noceti and Barbara Caputo and Claudio Castellini and Luca Baldassarre and Annalisa Barla and Lorenzo Rosasco and Francesca Odone and Giulio Sandini},
  title = {{Towards a Theoretical Framework for Learning Multi-modal Patterns for Embodied Agents}},
  booktitle = {ICIAP},
  year = {2009},
  doi = {10.1007/978-3-642-04146-4\_27},
}

Loading related content...

Figure