In the fields of computer vision and character animation, high-quality human motion data is essential. However, motion databases often suffer from inherent redundancy, where researchers capture identical movements multiple times. This leads to database bloat and increased computational costs without adding informative value. The Semantic Data Acquisition System (SDAS), presented at AAAI-25, addresses this by introducing a real-time framework to assess motion similarity semantically rather than just geometrically.
System Architecture
The SDAS framework is an end-to-end solution consisting of three primary components designed to streamline the data acquisition pipeline:
- Markerless MoCap System: Utilizing four synchronized RGB cameras, the system estimates 3D human poses using the SMPL (Skinned Multi-Person Linear) model, eliminating the need for specialized markers or suits.
- Real-time Data Storage Server: To facilitate instantaneous redundancy checks, the system employs an in-memory Redis database. It utilizes a K-Nearest Neighbor (K-NN) algorithm to retrieve the most similar existing motion from the database within milliseconds.
- Web-based Data Assessment Server: A Flask-based interface allows researchers to visualize new captures alongside retrieved data. It integrates Large Language Models (LLMs), specifically GPT-4, to provide natural language descriptions of the motion's characteristics, assisting in the decision to save or discard data.
The Semantic Similarity Metric
The core innovation of SDAS is its semantic similarity metric. Traditional methods often rely on raw joint coordinates (geometric similarity), which are sensitive to spatial transformations like orientation or speed. SDAS incorporates three high-level "semantic features":
- Stability: Measures the balance and groundedness of the pose.
- Liveliness: Captures the energy or intensity of the movement.
- Attention: Reflects the directional focus or intent of the motion.
By weighting these traits alongside raw pose parameters, the system can distinguish between a "brisk walk" and a "tired walk," recognizing them as semantically distinct even if the underlying geometry is similar.
Experimental Results and Robustness
The system was evaluated against the CMU MoCap database and new custom captures. A key strength of the SDAS metric is its robustness to spatial and temporal transformations. When compared against standard cosine similarity (Cossim), the proposed metric demonstrated significantly higher stability when motions were modified.
As shown in the evaluation, standard cosine similarity drops to 0.6271 when a motion is flipped (e.g., a left-handed vs. right-handed action), whereas the SDAS metric maintains a high similarity score of 0.8973. This confirms that the system recognizes the underlying action regardless of orientation.
In terms of retrieval accuracy, the system achieved a Top-1 Accuracy of 1.000 for fundamental actions such as "Walk," "Kick," and "Sit." Even for more complex, high-speed actions like "Punch" and "Throw," the system maintained respectable accuracies of 0.800 and 0.833, respectively.
Conclusion
SDAS provides a robust framework for building diverse motion datasets by minimizing redundancy through semantic analysis. By integrating markerless capture, high-speed retrieval, and LLM-based feedback, it empowers researchers to make informed decisions during the data acquisition process, ensuring that every saved motion adds unique value to the database.
Summary updated