SMamDiff: Spatial Mamba for Stochastic Human Motion Prediction

Junqiao Fan, Pengfei Liu, Haocong Rao

Abstract

With intelligent room-side sensing and service robots widely deployed, human motion prediction (HMP) is essential for safe, proactive assistance. However, many existing HMP methods either produce a single, deterministic forecast that ignores uncertainty or rely on probabilistic models that sacrifice kinematic plausibility. Diffusion models improve the accuracy-diversity trade-off but often depend on multi-stage pipelines that are costly for edge deployment. This work focuses on how to ensure spatial-temporal coherence within a single-stage diffusion model for HMP. We introduce SMamDiff, a Spatial Mamba-based Diffusion model with two novel designs: (i) a residual-DCT motion encoding that subtracts the last observed pose before a temporal DCT, reducing the first DC component ($f=0$) dominance and highlighting informative higher-frequency cues so the model learns how joints move rather than where they are; and (ii) a stickman-drawing spatial-mamba module that processes joints in an ordered, joint-by-joint manner, making later joints condition on earlier ones to induce long-range, cross-joint dependencies. On Human3.6M and HumanEva, these coherence mechanisms deliver state-of-the-art results among single-stage probabilistic HMP methods while using less latency and memory than multi-stage diffusion baselines.

Introduction

Human Motion Prediction (HMP) is the "anticipatory" engine for service robots and smart environments. To interact safely, a system needs to know not just where a person is, but where they might go next. Historically, researchers faced a trade-off: deterministic models are fast but produce a single "average" future; traditional probabilistic models offer variety but often sacrifice kinematic plausibility; and high-fidelity Diffusion Models (DMs) typically rely on heavy, multi-stage pipelines that are too slow for edge hardware.

SMamDiff addresses these challenges by creating a single-stage diffusion model that maintains high realism and diversity without computational bloat. It achieves this through two primary innovations: Residual-DCT encoding for temporal dynamics and a "stickman-drawing" Spatial Mamba for skeletal coordination.

Methodology

1. Residual-DCT Domain Diffusion

Most HMP models operate in the frequency domain using the Discrete Cosine Transform (DCT). However, standard DCT is often dominated by the "DC component" ($f=0$), which represents the average position. This causes models to "memorize" static locations rather than learning movement dynamics.

The Residual-DCT transformation process highlights movement dynamics by subtracting the last observed pose to focus on high-frequency velocity and acceleration cues.

SMamDiff subtracts the last observed pose from the entire sequence before applying the DCT. This "Residual-DCT" rebalances the data, forcing the model to focus on high-frequency cues like joint velocity and acceleration. This ensures the model learns the physics of the movement rather than just snapping to likely poses.

2. Stickman-Drawing Spatial Mamba

Standard models often use Graph Convolutional Networks (GCNs) to model the human skeleton, but GCNs are typically limited to immediate joint neighbors. SMamDiff instead utilizes a Mamba architecture—a state-space model efficient at handling long sequences—to "scan" joints in a specific "stickman-drawing" order (e.g., head $\rightarrow$ torso $\rightarrow$ limbs).

Overview of the SMamDiff architecture, including the diffusion process, the U-Net structure, and the internal Mam-Trans block.

By processing joints sequentially and backtracking through the skeleton, later joints condition on earlier ones. This allows the model to capture long-range dependencies, such as the coordination between a hand and a foot during a stride, which is often missed by local graph convolutions.

3. K-Diversity Loss

To prevent "mode collapse," where a model predicts the same future every time, SMamDiff uses a multi-head approach. During training, the model generates $K$ different hypotheses, but the loss is only calculated for the one that best matches the ground truth. This encourages different "heads" to specialize in distinct types of possible futures.

Results and Discussion

SMamDiff was evaluated on the Human3.6M and HumanEva-I benchmarks, outperforming existing single-stage probabilistic methods across several metrics:

  • Realism: On Human3.6M, it achieved a 37.3% improvement in FID (Fréchet Inception Distance) and a 27.4% improvement in CMD (Geometric Consistency) compared to the previous state-of-the-art, CoMusion.
  • Accuracy: It achieved the lowest Average Displacement Error (ADE) and Final Displacement Error (FDE) among compared methods, indicating its predictions stay closer to actual human movement over time.
  • Efficiency: The model uses a significantly smaller parameter count than multi-stage baselines (e.g., 4 Mamba blocks vs. 8 Transformer layers in HumanMAC), resulting in lower latency suitable for real-time robotics.

Conclusion

SMamDiff demonstrates that high-quality, diverse human motion prediction can be achieved within a streamlined, single-stage diffusion framework. By prioritizing dynamics over statics and skeletal logic over local graph constraints, it provides a robust solution for edge-deployed intelligent systems. Future work may involve exploring learnable joint-ordering sequences to better capture complex, non-linear actions.

Summary updated

Abstract

With intelligent room-side sensing and service robots widely deployed, human motion prediction (HMP) is essential for safe, proactive assistance. However, many existing HMP methods either produce a single, deterministic forecast that ignores uncertainty or rely on probabilistic models that sacrifice kinematic plausibility. Diffusion models improve the accuracy-diversity trade-off but often depend on multi-stage pipelines that are costly for edge deployment. This work focuses on how to ensure spatial-temporal coherence within a single-stage diffusion model for HMP. We introduce SMamDiff, a Spatial Mamba-based Diffusion model with two novel designs: (i) a residual-DCT motion encoding that subtracts the last observed pose before a temporal DCT, reducing the first DC component ($f=0$) dominance and highlighting informative higher-frequency cues so the model learns how joints move rather than where they are; and (ii) a stickman-drawing spatial-mamba module that processes joints in an ordered, joint-by-joint manner, making later joints condition on earlier ones to induce long-range, cross-joint dependencies. On Human3.6M and HumanEva, these coherence mechanisms deliver state-of-the-art results among single-stage probabilistic HMP methods while using less latency and memory than multi-stage diffusion baselines.
BibTeX
@misc{fan2025smamdiff,
  author = {Junqiao Fan and Pengfei Liu and Haocong Rao},
  title = {{SMamDiff: Spatial Mamba for Stochastic Human Motion Prediction}},
  year = {2025},
  eprint = {2512.00355},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.00355},
}
Figure