Abstract
High-fidelity motion capture (mocap) is essential for immersive Metaverse experiences, yet the resulting high-frequency data streams often exceed network capacities, causing perceptible latency. This paper introduces SIDQL (Spherical Interpolation based Deep Q-Learning), a framework that optimizes data transmission by intelligently selecting keyframes and reconstructing intermediate motion. By employing a spherical coordinate system to preserve physical constraints and a Deep Q-Learning agent to automate keyframe selection, SIDQL achieves a decision latency of 0.004s and maintains low reconstruction errors.
Introduction
In the Metaverse, digital avatars must mirror real-world movements with minimal lag. Current mocap systems generate massive amounts of data, necessitating efficient compression techniques. Traditional methods often rely on keyframe extraction followed by interpolation in Cartesian space. However, these approaches face two primary challenges: the "stretchy bone" problem, where linear interpolation in 3D space alters bone lengths, and subjective keyframe selection based on human-labeled datasets rather than mathematical optimality.
Methodology
The SIDQL framework addresses these issues through a dual-pronged approach involving physics-informed geometry and reinforcement learning.
1. Spherical Reconstruction Algorithm
To eliminate the "stretchy bone" artifact, SIDQL converts motion data from Cartesian $(x, y, z)$ coordinates to a spherical coordinate system $(r, \theta, \phi)$. By treating the bone length ($r$) as a constant, the framework ensures that limbs do not unnaturally stretch or shrink during reconstruction. The system uses third-order polynomial functions to interpolate the angular components ($\theta, \phi$) and the root point's trajectory, ensuring fluid transitions between keyframes.
2. Deep Q-Learning (DQL) for Keyframe Extraction
Rather than relying on human-labeled data, SIDQL utilizes a DQL agent to identify optimal keyframes. The agent's reward function is tied directly to the minimization of the Mean Angle Error (MAE) across the reconstructed sequence. This allows the model to learn mathematically optimal frame selections that maximize reconstruction accuracy.
Evaluation and Results
The framework was evaluated using the CMU Graphics Lab Motion Capture Database, focusing on 23 major joints downsampled to 30 FPS.
- Accuracy: For a 60-frame sequence, extracting just 5 keyframes resulted in a reconstruction error of less than 0.09.
- Efficiency: SIDQL demonstrated a decision time of 0.004 seconds per frame, making it 225 times faster than traditional "Greedy" optimization algorithms (0.9s).
- Performance: The model significantly outperformed baseline methods like random or uniform selection, particularly in non-linear movements such as jumping and running.
Conclusion
SIDQL provides a robust solution for real-time motion capture in bandwidth-constrained environments. By integrating physical constraints into the coordinate system and automating keyframe selection via RL, the framework balances high fidelity with ultra-low latency. Future work will focus on addressing artifacts in high-velocity movements and deploying the system on physical hardware.
Summary updated