Towards Arbitrary Motion Completing via Hierarchical Continuous Representation

Chenghao Xu, Guangtao Lyu, Qi Liu, Jiexi Yan, Muli Yang, Cheng DengarXiv 2025 · DOI · arXiv

Introduction

In computer graphics and robotics, human movement is traditionally captured and stored as discrete sequences of frames at fixed rates, such as 30 or 60 FPS. While this discretization is computationally convenient, it creates a significant "fidelity gap." When motion data is slowed down or converted between frame rates, traditional interpolation methods often produce jittery, physically unrealistic, or "mushy" results.

To bridge this gap, researchers have looked toward Implicit Neural Representations (INRs), which treat data as continuous functions. However, standard INRs—originally designed for static images or video pixels—often struggle with the high-frequency temporal nuances of human motion, such as the rapid acceleration of a limb or the subtle vibrations of a foot strike. This paper introduces PA-HiRes (Parametric Activation-induced Hierarchical Implicit Representation), a novel framework that treats human motion as a continuous mathematical function, enabling high-fidelity completion and interpolation at any arbitrary frame rate.

Methodology

PA-HiRes addresses the limitations of existing INRs through two primary architectural innovations: Multi-Scale Temporal Encoding and Parametric Activation Induced Decoding.

Multi-Scale Temporal Encoding (MSTE)

Human motion is inherently multi-scale; a single action like walking involves broad, low-frequency movements of the torso alongside rapid, high-frequency movements of the extremities. PA-HiRes utilizes Multi-Scale Temporal Encoding (MSTE) to extract features from motion clips at various temporal resolutions. By processing the input through a hierarchy of scales, the model can simultaneously capture the global context of a motion sequence and the intricate details of specific gestures.

Parametric Activation Induced Decoding (PAID)

Standard neural networks typically rely on activation functions like ReLU. While effective for many tasks, these functions suffer from "spectral bias," meaning they prioritize learning smooth, low-frequency patterns and struggle to reconstruct sharp, high-frequency details.

To overcome this, the authors propose Parametric Activation Induced Decoding (PAID). Inspired by Kolmogorov-Arnold Networks (KAN), PAID replaces static activations with a learnable Fourier-based function:

$$\sigma(x) = \sum_{i=1}^{\zeta} \Psi_i \sin(\Omega_i x + \Phi_i)$$

By representing the activation as a sum of sine waves with learnable parameters, the network effectively becomes a frequency-domain tool. This allows PA-HiRes to dynamically adapt its expressiveness to the specific complexity of the motion, capturing the subtle "wiggles" and high-frequency nuances that ReLU-based models often smooth away.

Training Objectives

The framework is optimized using a combination of Mean Squared Error (MSE) for joint positions and a velocity consistency constraint. This constraint ensures that the derived motion is not only visually smooth but also physically plausible by maintaining temporal coherence across the continuous time domain.

Experimental Evaluation

The model was evaluated on three gold-standard 3D human motion benchmarks:

  • HumanML3D: A large-scale dataset (20 FPS).
  • LaFAN1: Focused on high-quality transitions (30 FPS).
  • CMU Mocap: A high-fidelity dataset (120 FPS).

Key Results

1. State-of-the-Art Interpolation PA-HiRes consistently outperformed existing INR methods, such as LIIF and Meta-SR. On the HumanML3D dataset, the model achieved a PSNR of 36.982 for 2x interpolation, a significant improvement over the previous state-of-the-art of 34.734.

2. Robust Non-Integer Scaling Unlike traditional interpolation techniques that are often limited to integer factors (e.g., 2x or 4x), PA-HiRes can query any time coordinate $t$. This allows for arbitrary, non-integer scaling (e.g., 1.2x or 4.8x) without introducing structural artifacts or losing fidelity.

3. Motion In-Betweening and Extrapolation In tasks requiring the model to "fill in the blanks" between distant poses, PA-HiRes outperformed specialized architectures like TS-Former, showing lower errors in both global position (L2P) and rotation (L2Q). Furthermore, the model demonstrated a robust ability to extrapolate, predicting plausible continuations of movement outside the original input range.

Discussion and Future Work

The success of PA-HiRes highlights the importance of hierarchical modeling and frequency-aware activation functions in representing human motion. Ablation studies confirmed that increasing the number of temporal scales ($S$) significantly boosts performance, validating the multi-scale nature of human movement.

However, the authors note certain limitations. There is an inherent trade-off between the number of Fourier terms ($\zeta$), temporal scales ($S$), and computational overhead. Additionally, while the velocity loss improves physical plausibility, the model does not yet explicitly account for complex physical constraints like gravity, floor contact (to prevent foot sliding), or joint torque limits.

Future research may involve integrating this continuous representation into large-scale text-to-motion generative models, such as Motion Diffusion Models. This would allow generative systems to produce motion at any desired frame rate natively, further closing the gap between discrete digital representations and the continuous reality of human movement.

Summary updated

Abstract

Physical motions are inherently continuous, and higher camera frame rates typically contribute to improved smoothness and temporal coherence. For the first time, we explore continuous representations of human motion sequences, featuring the ability to interpolate, inbetween, and even extrapolate any input motion sequences at arbitrary frame rates. To achieve this, we propose a novel parametric activation-induced hierarchical implicit representation framework, referred to as NAME, based on Implicit Neural Representations (INRs). Our method introduces a hierarchical temporal encoding mechanism that extracts features from motion sequences at multiple temporal scales, enabling effective capture of intricate temporal patterns. Additionally, we integrate a custom parametric activation function, powered by Fourier transformations, into the MLP-based decoder to enhance the expressiveness of the continuous representation. This parametric formulation significantly augments the model's ability to represent complex motion behaviors with high accuracy. Extensive evaluations across several benchmark datasets demonstrate the effectiveness and robustness of our proposed approach.
BibTeX
@misc{xu2025towards,
  author = {Chenghao Xu and Guangtao Lyu and Qi Liu and Jiexi Yan and Muli Yang and Cheng Deng},
  title = {{Towards Arbitrary Motion Completing via Hierarchical Continuous Representation}},
  year = {2025},
  eprint = {2512.21183},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.21183},
}
Figure