Abstract
Optical motion capture (mocap) is the industry standard for high-fidelity human movement tracking, yet it produces raw 3D point clouds that require labor-intensive manual labeling. We introduce SOMA, a transformer-based framework that automates this process without subject-specific calibration. By training exclusively on synthetic data generated from the AMASS dataset, SOMA learns to map unordered, noisy point clouds to anatomical labels. Our approach utilizes an optimal transport layer to enforce physical constraints, achieving F1 scores of 95-98% on real-world data and outperforming existing state-of-the-art methods, particularly in the presence of occlusions.
Introduction
The "dark matter" of motion capture refers to the vast quantities of archival, unlabeled 3D point clouds that remain unusable because the original subject calibration data is lost. Traditional commercial tools rely on these calibrations to assign labels (e.g., "left elbow") to raw coordinates. When markers are occluded or spurious "ghost" points appear, these systems often fail, necessitating hours of manual cleanup. SOMA addresses these challenges by treating labeling as a constrained matching problem, providing a robust, automated solution for both live and archival data.
Methods
SOMA's architecture is built on two core pillars: a deep transformer network and an optimal transport layer.
Transformer-Based Spatial Learning
Instead of traditional convolutional layers, SOMA employs stacked self-attention layers. This allows the model to learn complex spatial relationships between markers. As the data passes through deeper layers, the network naturally focuses on geodesically proximal body regions, effectively reconstructing the human skeletal structure from a set of unordered dots.
Optimal Transport and "Dustbins"
To ensure a physically plausible one-to-one mapping between points and labels, SOMA incorporates an optimal transport layer with Sinkhorn normalization. A key innovation is the introduction of "dustbins," which allow the model to categorize spurious reflections or "ghost" points as outliers rather than mislabeling them as body parts.
Synthetic Training Pipeline
A significant hurdle in mocap research is the scarcity of labeled real-world training data. SOMA overcomes this by training entirely on synthetic data. Using the AMASS dataset to animate SMPL-X body models, the authors simulated realistic mocap artifacts—such as marker displacement and trajectory breaks—to ensure the model generalizes to real-world noise.
Results and Discussion
SOMA was evaluated against several benchmark datasets (BMLrub, KIT, BMLmovi) and compared directly with high-end commercial software like Vicon Shogun.
Performance and Robustness
SOMA consistently achieved F1 scores between 95% and 98%. Notably, it maintains high accuracy even as the number of occluded markers increases, a scenario where traditional methods typically degrade.
Archival Data Rescue
The framework's calibration-free nature allowed the authors to process over 8 hours of previously unusable archival data from datasets like Mixamo and DanceDB. This demonstrates SOMA's potential to unlock historical motion data for modern animation and biomechanical analysis.
Conclusion
SOMA represents a significant step toward fully automated, calibration-free motion capture. By combining the representational power of Transformers with the physical constraints of Optimal Transport, the model provides a robust solution to the labeling problem. Future work will focus on integrating temporal information to improve consistency across frames and extending the framework to multi-person tracking scenarios.
Summary updated