Semantic Clusters Combined with Kinematics: The Case of English and Modern Greek Motion Verbs

Marietta Sionti, Leonardo Claudino, Yiannis Aloimonos, Carolyn Penstein Rosé, Στέλλα ΜαρκαντωνάτουOpenalex Unknown 2014 · DOI · OpenAlex

Abstract

We combine corpus driven linguistic knowledge with experimentally obtained sensorimotor data in an effort to better specify the minimum conceptual representation of a motion event that distinguishes it from all other events, which still are largely presented in a vague and not objectively calculated quantitative method. We use American English and Modern Greek data as a case study, in order to focus on the clustering of motor actions and its correspondence to previous linguistic classifications of both languages.

Introduction

In linguistics, motion verbs (e.g., "run," "crawl," "jump") are traditionally classified based on subjective criteria or corpus data—essentially, how we talk about moving. However, these classifications often lack an objective, quantitative foundation. While we intuitively know "walking" differs from "staggering," defining the minimum physical information required to distinguish them remains a challenge.

This research addresses the problem of perceptual grounding: connecting abstract linguistic notions like directionality and transitivity to sensorimotor data. By grounding motion verbs in high-fidelity 3D kinematic data, the authors aim to create robust semantic classifications. Such work has significant implications for Natural Language Processing (NLP), machine translation, and human-robot interaction.

Methodology

The study employed a multi-stage approach combining linguistic theory with motion capture technology. The researchers selected 25 core motion predicates from established Greek and English classifications (e.g., Antonopoulou for Greek; Levin and FrameNet for English).

Experimental setup showing a subject wearing the Xsens Moven suit within the capture area.

To record these actions, eight native Modern Greek speakers and eight native American English speakers (aged 25–30) performed the selected verbs while wearing an Xsens Moven suit. This full-body system utilizes 16 inertial trackers equipped with 3D gyroscopes and accelerometers to track 23 segments and 22 joints, providing absolute orientation and eliminating data "drift."

The resulting dataset of 3D kinematic sequences was normalized for distance and duration. To bridge the gap between languages, a video-based survey was conducted where native English speakers described the Greek participants' actions using the most natural English equivalents.

Kinematic Analysis and Results

The raw data, consisting of joint angles and orientations over time, was processed using Principal Component Analysis (PCA). This statistical method identified "synergies"—stable, latent patterns across many dimensions of low-level data—allowing complex 3D movements to be projected into a 2D space for visualization.

3D skeletal model illustrating the joint segments and coordinate axes used for kinematic tracking.

The analysis revealed that unsupervised statistical methods could replicate many aspects of manual linguistic classification:

  • Physical Clusters Mirror Linguistic Classes: PCA plots showed distinct groups, such as a "rectangular" cluster for leg-related actions (walking, marching) and a "blue ellipsis" cluster for arm-related manipulation (lifting, turning a page).
  • Morphology vs. Performance: In Greek, verbs sharing the same root (e.g., pidho for various types of jumping) formed tight physical groups despite different spatial outcomes, validating their linguistic grouping.
  • Variance Factors: The first Principal Component (PC1), accounting for 43-51% of variance, was primarily related to hip displacement on the sagittal plane. PC2 (16-25% of variance) related to knee height and shoulder movement.

The study also noted cultural divergences. For instance, English speakers used their arms less than Greek speakers when running. Additionally, some Greek verbs like dhraskelizo (stride) required physical obstacles for correct performance, whereas English speakers distinguished "stride" from "step over" more abstractly.

Conclusion and Future Work

This research demonstrates that physical "synergies" are a valid proxy for semantic meaning. However, limitations remain. Current sensorimotor methods cannot yet measure abstract criteria like "intentionality" (e.g., falling vs. jumping). Furthermore, the study used a limited set of "unmarked" objects; future work will explore how hand and body posture change based on the specific object being grasped. Scaling this proof-of-concept to a larger population will be necessary to develop a definitive kinematic "identification card" for every motion verb.

Summary updated

Abstract

We combine corpus driven linguistic knowledge with experimentally obtained sensorimotor data in an effort to better specify the minimum conceptual representation of a motion event that distinguishes it from all other events, which still are largely presented in a vague and not objectively calculated quantitative method. We use American English and Modern Greek data as a case study, in order to focus on the clustering of motor actions and its correspondence to previous linguistic classifications of both languages.
BibTeX
@inproceedings{sionti2014semantic,
  author = {Marietta Sionti and Leonardo Claudino and Yiannis Aloimonos and Carolyn Penstein Rosé and Στέλλα Μαρκαντωνάτου},
  title = {{Semantic Clusters Combined with Kinematics: The Case of English and Modern Greek Motion Verbs}},
  booktitle = {Openalex Unknown},
  year = {2014},
  doi = {10.2478/9788376560762.p318},
}
Figure