MambaHand: Local Identity and Behavioral Sequence Learning with Mamba for Dynamic Hand Gesture Authentication

Zenan Lin, Junduan Huang, Xilai Wang, Wenxiong KangarXiv 2026 · arXiv

TL;DR

MambaHand Net is a novel two-stream framework that combines the spatial precision of CNNs with the efficient long-range modeling of Mamba to authenticate users via dynamic hand gestures. By capturing both stable physiological traits (like hand shape) and unique behavioral "signatures" (like acceleration patterns), it achieves state-of-the-art performance on the SCUT-DHGA benchmark.

What Problem Does This Paper Address?

Dynamic Hand Gesture Authentication (DHGA) is a biometric challenge that asks "who is performing the gesture?" rather than "what gesture is being performed?" To answer this, a model must distinguish between two people performing the same motion by looking at fine-grained physical differences (hand texture, size) and behavioral habits (speed, rhythm, and acceleration).

Existing methods typically rely on 3D CNNs or manually designed motion representations. However, 3D CNNs are computationally expensive and often struggle with very long sequences, while manual representations may miss subtle behavioral nuances. The authors identify a need for a framework that can jointly capture local identity cues and long-range behavioral dependencies without the quadratic complexity of Transformers.

Methods

The authors introduce MambaHand Net, a two-stream architecture built upon a shared ResNet18 encoder. The framework splits the task into two specialized paths:

  1. Local Identity Stream (LIS): This stream uses standard convolutional layers to extract stable, frame-level physiological features such as hand shape and texture.
  2. Behavioral Sequential Stream (BSS): This is the core innovation. It treats intermediate visual features as "spatial tokens" and constructs two types of sequences:
  3. 1st-order Temporal Difference: Captures velocity-like variations (speed and direction).
  4. 2nd-order Temporal Difference: Captures acceleration and deceleration patterns.

These sequences are processed by Bidirectional Mamba (Bi-Mamba) blocks. Mamba’s selective state-space mechanism allows for efficient modeling of long-range dependencies with linear complexity, making it ideal for gesture videos.

Finally, the Attention-Guided Behavioral Token Aggregation (ABTA) module (see Figure 5) adaptively weights these tokens to highlight the most discriminative regions of the hand during motion, before fusing both streams into a final identity representation.

Fig. 1. Overall architecture of the proposed MambaHand Net. The framework consists of a LIS and a BSS built upon a shared encoder. The LIS extracts frame-level local identity cues, while the BSS preserves spatial tokens from intermediate features and constructs first-order and second-order temporal difference sequences for behavioral modeling. Two Bi-Mamba-based branches are employed to capture multi-order behav- ioral dynamics, followed by ABTA, branch fusion, and two-stream feature fusion for final identity representation learning.

Datasets

The model was evaluated on the SCUT-DHGA dataset, a large-scale benchmark containing 29,160 gesture videos from 193 subjects. The authors tested the model under two primary protocols:

  • MG (Defined Gesture): The model is trained on all gestures and tested on a specific gesture.
  • UMG (Unseen Gesture): The model is trained on a subset of gestures and tested on a completely new gesture, evaluating its ability to generalize behavioral patterns to unseen motions.

Key Insights and Ideas

  • The "Sweet Spot" for Features: Through ablation studies, the authors found that connecting the Behavioral Stream to the Layer 2 features of the ResNet encoder provided the best results. Shallow features (Layer 1) had too much noise, while deeper features (Layer 4) lost the fine-grained local motion cues necessary for authentication.
  • Mamba vs. Transformers: While Transformers are popular for sequences, Mamba’s linear complexity makes it significantly more efficient for the long-range dependencies found in hand gestures, where a "habit" might span the entire video.
  • Multi-Order Dynamics: Simply looking at frame-to-frame changes isn't enough. By explicitly modeling 2nd-order differences (acceleration), the model captures the unique "jerk" or smoothness of a specific user's movement.
  • Task-Specific Design Matters: The authors noted that directly applying vanilla Mamba models (like Vision Mamba) yielded poor results. The success of MambaHand Net comes from the collaborative design of the CNN encoder for identity and the Mamba-based BSS for behavior.

Key Results

MambaHand Net achieved an average Equal Error Rate (EER) of 0.984%, outperforming previous state-of-the-art methods like TS-DHGANet and various 3D CNN architectures.

Table 2. Comparison of our approach with other networks.

As shown in Table 2 (Figure 3), the model is particularly robust in the challenging "Cross-Session" setting, where enrollment and verification videos are recorded a week apart. It maintains a competitive parameter count (13.7M) and computational cost (38.4 GFLOPs), making it more practical than many heavy 3D CNN alternatives.

Limitations and Future Work

While MambaHand Net shows excellent performance, the authors point toward several areas for improvement:

  • Efficiency: Investigating even more lightweight Mamba architectures for edge devices.
  • Multi-modality: Integrating other sensors (like depth or infrared) to improve robustness against spoofing.
  • Deployment: Moving toward real-time mobile deployment to test the system in real-world authentication scenarios.

The paper demonstrates that the "CNN for space, Mamba for time" paradigm is a powerful new direction for biometric video analysis.

Summary updated

Abstract

Dynamic hand gesture authentication (DHGA) is an emerging biometric technique that contains both physiological and behavioral identity characteristics. However, existing methods primarily rely on convolutional networks or manually designed motion representations, making it difficult to jointly capture fine-grained local identity differences and long-range behavioral dependencies during hand gesture execution. To address these challenges, we propose a two-stream identity representation learning framework, MambaHand Net, to combine CNNs with Mamba for DHGA. The proposed framework consists of a Local Identity Stream (LIS) and a Behavioral Sequential Stream (BSS). The LIS extracts frame-wise local identity cues with a convolutional network, while the BSS preserves spatial tokens from intermediate visual features and constructs first-order and second-order temporal difference sequences to model multi-order behavioral variations. Moreover, the bidirectional Mamba is incorporated into the BSS to model semantic motion across different spatial locations, enabling efficient long-range modeling of hand gesture execution habits. Finally, attention-guided spatial token aggregation and a two-stream fusion strategy are used to integrate local identity cues and multi-order behavioral patterns into a discriminative identity representation. Our method achieves state-of-the-art (SOTA) average performance on the SCUT-DHGA benchmark dataset, demonstrating that combining CNNs with Mamba is an effective new solution for DHGA.
BibTeX
@misc{lin2026mambahand,
  author = {Zenan Lin and Junduan Huang and Xilai Wang and Wenxiong Kang},
  title = {{MambaHand: Local Identity and Behavioral Sequence Learning with Mamba for Dynamic Hand Gesture Authentication}},
  year = {2026},
  eprint = {2611.04626},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04626},
}
Figure