Introduction
For individuals with amputations or mobility-impairing conditions, virtual reality (VR) rehabilitation and prosthetic control often require the generation of kinematic data for limbs that are missing or immobile. The computational challenge lies in inferring the movement of these "phantom limbs" in real-time based on the movement of functional body parts. Existing models like RNNs or GANs often fail to meet real-time latency requirements or require massive datasets. This paper introduces a Visible Hierarchical Neural Network (VHNN) based on parallelized autoencoders to address these challenges.
Methodology: Visible Hierarchical Neural Networks
The proposed VHNN architecture leverages the natural biomechanical correlations of the human body. Instead of a monolithic network, the authors model the body as a hierarchical "forest" structure based on functional correlations.
Musculoskeletal Mapping and Parallelization
The VHNN splits the input tensor of functional joint data into multiple smaller, parallelized 1D-convolutional autoencoders. Each sub-network is responsible for a specific joint segment where motion is mostly non-divergent. This anatomy-informed architecture prevents the network from becoming "confused" by opposing limb movements, a phenomenon the authors call "optimization bias."
The authors evaluated two variants:
- VHNN-2: Utilizes two subsystems.
- VHNN-4: Utilizes four subsystems with cross-correlation pipelines, allowing inputs from both arms to influence the generated outputs of both legs, enhancing robustness to user movement errors.
Results and Discussion
The VHNN architectures were trained on a small dataset of virtual Tai-Chi master kinetic data (less than 25 samples per motion). Despite the limited data, the models significantly outperformed traditional Multilayer Perceptron (MLP) and Denoising Autoencoder (DAE) baselines.
- Computational Efficiency: VHNN-2 achieved a per-step prediction time of 30–31 μs, significantly faster than the standard DAE (108–110 μs).
- Accuracy: VHNN-2 reduced ground truth error from 2107.46 (DAE) to 276.68.
- Real-World Robustness: While VHNN-2 was more accurate on paper, VHNN-4 proved more effective in live tests due to its tolerance for "patient error"—instances where the user does not perfectly replicate the target movement.
Conclusion and Future Work
By mirroring the physical musculoskeletal hierarchy, the VHNN achieves high-fidelity phantom limb generation with minimal data and low latency. Future research will focus on mitigating architectural noise in complex variants like VHNN-4, exploring GANs for longer-term prediction, and conducting clinical studies to evaluate the psychological benefits for patients, such as the reduction of phantom limb pain.
Summary updated