Grounded Representations for Sensory-Motor Learning: A Linguistic Approach

Yiannis Aloimonos, Gutemberg Guerra‐FilhoeScholarship 2006

Abstract

We have empirically discovered that the space of human actions has a linguistic structure.This is a sensory-motor space consisting of the evolution of the joint angles of the human body in movement.The space of human activity has its own phonemes, morphemes, words (nouns, verbs, adjectives, adverbs), and sentences formed by syntax.This has a number of implications for the grounding problem and cognition in general.We present a Human Activity Language (HAL) for symbolic manipulation of visual and motor information.The embodiment of the language serves as the interface between visual perception and the motor subsystem.The visuo-motor language is defined using a linguistic approach.In phonology, we define basic atomic segments that are used to compose human activity.In morphology, we study how visuo-motor phonemes are combined to form strings representing human activity and to generate a higher-level morphological grammar.In syntax, we present a model for visuo-motor sentence construction where the subject corresponds to the active joints (noun) modified by a posture (adjective).A verbal phrase involves the representation of the human activity (verb) and timing coordination among different joints (adverb).
BibTeX
@inproceedings{aloimonos2006grounded,
  author = {Yiannis Aloimonos and Gutemberg Guerra‐Filho},
  title = {{Grounded Representations for Sensory-Motor Learning: A Linguistic Approach}},
  booktitle = {eScholarship},
  year = {2006},
}

Loading related content...

Figure