Adaptive Generation of Phantom Limbs Using Visible Hierarchical Autoencoders

Dakila Ledesma, Yu Liang, Dalei WuarXiv 2019 · arXiv

Introduction

For individuals with amputations or mobility-impairing conditions, virtual reality (VR) rehabilitation and prosthetic control often require the generation of kinematic data for limbs that are missing or immobile. The computational challenge lies in inferring the movement of these "phantom limbs" in real-time based on the movement of functional body parts. Existing models like RNNs or GANs often fail to meet real-time latency requirements or require massive datasets. This paper introduces a Visible Hierarchical Neural Network (VHNN) based on parallelized autoencoders to address these challenges.

Methodology: Visible Hierarchical Neural Networks

The proposed VHNN architecture leverages the natural biomechanical correlations of the human body. Instead of a monolithic network, the authors model the body as a hierarchical "forest" structure based on functional correlations.

Hierarchical mapping of kinetic degrees of freedom (DOFs) from functional body parts to phantom limbs.

Musculoskeletal Mapping and Parallelization

The VHNN splits the input tensor of functional joint data into multiple smaller, parallelized 1D-convolutional autoencoders. Each sub-network is responsible for a specific joint segment where motion is mostly non-divergent. This anatomy-informed architecture prevents the network from becoming "confused" by opposing limb movements, a phenomenon the authors call "optimization bias."

Technical architecture of the VHNN, showing parallelized 1D-convolutional autoencoder pipelines.

The authors evaluated two variants:

  • VHNN-2: Utilizes two subsystems.
  • VHNN-4: Utilizes four subsystems with cross-correlation pipelines, allowing inputs from both arms to influence the generated outputs of both legs, enhancing robustness to user movement errors.

Results and Discussion

The VHNN architectures were trained on a small dataset of virtual Tai-Chi master kinetic data (less than 25 samples per motion). Despite the limited data, the models significantly outperformed traditional Multilayer Perceptron (MLP) and Denoising Autoencoder (DAE) baselines.

  • Computational Efficiency: VHNN-2 achieved a per-step prediction time of 30–31 μs, significantly faster than the standard DAE (108–110 μs).
  • Accuracy: VHNN-2 reduced ground truth error from 2107.46 (DAE) to 276.68.
  • Real-World Robustness: While VHNN-2 was more accurate on paper, VHNN-4 proved more effective in live tests due to its tolerance for "patient error"—instances where the user does not perfectly replicate the target movement.

Conclusion and Future Work

By mirroring the physical musculoskeletal hierarchy, the VHNN achieves high-fidelity phantom limb generation with minimal data and low latency. Future research will focus on mitigating architectural noise in complex variants like VHNN-4, exploring GANs for longer-term prediction, and conducting clinical studies to evaluate the psychological benefits for patients, such as the reduction of phantom limb pain.

Summary updated

Abstract

This paper proposed a hierarchical visible autoencoder in the adaptive phantom limbs generation according to the kinetic behavior of functional body-parts, which are measured by heterogeneous kinetic sensors. The proposed visible hierarchical autoencoder consists of interpretable and multi-correlated autoencoder pipelines, which is directly derived from the hierarchical network described in forest data-structure. According to specified kinetic script (e.g., dancing, running, etc.) and users' physical conditions, hierarchical network is extracted from human musculoskeletal network, which is fabricated by multiple body components (e.g., muscle, bone, and joints, etc.) that are bio-mechanically, functionally, or nervously correlated with each other and exhibit mostly non-divergent kinetic behaviors. Multi-layer perceptron (MLP) regressor models, as well as several variations of autoencoder models, are investigated for the sequential generation of missing or dysfunctional limbs. The resulting kinematic behavior of phantom limbs will be constructed using virtual reality and augmented reality (VR/AR), actuators, and potentially controller for a prosthesis (an artificial device that replaces a missing body part). The addressed work aims to develop practical innovative exercise methods that (1) engage individuals at all ages, including those with a chronic health condition(s) and/or disability, in regular physical activities, (2) accelerate the rehabilitation of patients, and (3) release users' phantom limb pain. The physiological and psychological impact of the addressed work will critically be assessed in future work.
BibTeX
@misc{ledesma2019adaptive,
  author = {Dakila Ledesma and Yu Liang and Dalei Wu},
  title = {{Adaptive Generation of Phantom Limbs Using Visible Hierarchical Autoencoders}},
  year = {2019},
  eprint = {1910.01191},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1910.01191},
}
Figure