Abstract
In-air signature verification is vital for biometric user identification in contact-less mode. The state-of-the-art methods use heuristics for signature acquisition, and provide insufficient data to train neural networks for the verification. In this article, we present a novel method for end-to-end deep learning based in-air signature verification using a depth sensor. In this regard, we propose a new medium-scale in-air signature dataset which is created using an accurate convolutional neural network (CNN) based 3D hand pose estimation algorithm. The proposed dataset offers a total of 1800 signatures collected from 40 subjects. So far, dynamic time warping (DTW) has been the most effective and commonly used method for verification. Keeping in view the significant advancement in deep learning, we present a more accurate deep learning based alternative which outperforms DTW by 67.6%. To this end, we train a personalized autoencoder to reconstruct the signature of each subject. Thereafter, the signature is verified by thresholding the reconstruction loss. We perform extensive experiments by formulating spatial and depth features of the signature in images and point clouds based representations. Moreover, for comparisons, we implement several deep learning algorithms (i.e. linear autoencoder, convolutional autoencoder, and Deep One-Class classifier). Our verification approach achieves an EER of 0.055%.
Introduction
As virtual reality (VR), augmented reality (AR), and smart environments become mainstream, the need for reliable, contactless biometric verification has grown. In-air signature verification—where a user signs their name in the air using their finger—is a promising solution. However, traditional methods have been limited by inaccurate tracking based on brittle heuristics and a lack of large-scale datasets suitable for deep learning.
DeepAirSig addresses these challenges by introducing an end-to-end deep learning framework that tracks the user's index finger in 3D space using a standard depth camera and verifies identity using personalized autoencoders. By accurately tracking 3D coordinates, the system achieves a remarkable 0.055% Equal Error Rate (EER).
Methodology
The proposed pipeline modernizes both the acquisition and verification stages of in-air signatures.
Robust 3D Acquisition
Instead of relying on simple computer vision heuristics, the system employs a Convolutional Neural Network (CNN) trained on the million-scale BigHand2.2m dataset. This model accurately estimates the 3D coordinates of the index fingertip in every frame, even during fast movements or partial occlusions.
Signature Representations
The raw 3D trajectory is smoothed and normalized, then evaluated using two distinct formats to determine the most effective input for deep learning:
- Image-based: The 3D trajectory is projected onto a 2D image plane, with depth encoded as grayscale pixel intensity.
- PointCloud-based: The trajectory is treated as a 1D sequence of 3D points (X, Y, Z coordinates).
Verification via Personalized Autoencoders
Verification is treated as an anomaly detection problem. A personalized autoencoder is trained for each user using only their genuine signatures. During verification, the model attempts to reconstruct the input signature. If the reconstruction loss exceeds a specific threshold, the signature is flagged as a forgery. The study utilized Convolutional Autoencoders for image data and Linear Autoencoders for point cloud data, which naturally exploit the consistent sequence lengths of genuine signatures.
Experimental Results
The authors introduced the DeepAirSig dataset, containing 1,800 signatures from 40 subjects, to evaluate the system. The primary metric for evaluation was the Equal Error Rate (EER).
- Performance: The PointCloud-based verification using full 3D (X,Y,Z) features achieved a state-of-the-art 0.055% EER.
- Comparison: This result represents a 67.6% improvement over the traditional Dynamic Time Warping (DTW) baseline (0.170% EER) and outperformed Deep One-Class SVDD (0.207% EER).
- Representation Impact: The PointCloud-based approach significantly outperformed the Image-based representation (0.260% EER), suggesting that 2D projections introduce sub-optimal distortions for 3D trajectory data.
Conclusion
DeepAirSig demonstrates that deep learning, particularly when applied to 3D point cloud trajectories, can drastically improve the security and accuracy of in-air signature verification. By leveraging depth as a critical security feature, the system makes it significantly harder for imposters to forge signatures. Future work may involve scaling the dataset to thousands of users and exploring temporal models like Transformers to further capture the dynamics of 3D signatures.
Summary updated