DeepWriteSYN: On-Line Handwriting Synthesis via Deep Short-Term Representations

Ruben Tolosana, Paula Delgado-Santos, Andrés Pérez-Uribe, Rubén Vera-Rodríguez, Julian Fiérrez, Aythami Morales

Abstract

DeepWriteSYN is a novel framework designed to generate realistic on-line handwriting, such as signatures and digits, by decomposing them into short-term "strokes" and processing them through a sequence-to-sequence Variational Autoencoder (VAE). This approach addresses the critical challenge of data scarcity in biometric systems, where often only a single enrollment sample is available. By synthesizing natural intra-subject variations, DeepWriteSYN enables robust data augmentation. Experimental results demonstrate that incorporating these synthetic samples can reduce signature verification error rates by over 50%, significantly outperforming traditional synthesis methods.

Introduction

In biometric authentication, "on-line" handwriting refers to the time-sequence of $(x, y)$ coordinates captured during the writing process. While deep learning has advanced static image synthesis, generating realistic time-series handwriting data remains difficult due to the inherent variability of human movement and the scarcity of training data. In many real-world "one-shot" scenarios, a verification system may only have access to a single enrollment sample. DeepWriteSYN aims to bridge this gap by generating synthetic, natural variations that help models distinguish between genuine intra-subject variation and skillful forgeries.

Methodology

The DeepWriteSYN architecture consists of two primary modules:

  1. Temporal Segmentation: Rather than modeling a complete, complex signature, the system decomposes handwriting into "short-term" segments or strokes. This is achieved using a velocity-based profile; segments are defined by points where pen speed crosses statistical thresholds (mean and standard deviation).
Temporal segmentation process: an input signature is decomposed into strokes based on its velocity profile.
  1. Short-Term VAE: These segments are processed by a Variational Autoencoder based on the Sketch-RNN architecture. An encoder (Bidirectional RNN) compresses the stroke into a latent vector $z$, while an autoregressive decoder uses a Gaussian Mixture Model (GMM) to predict subsequent points.
DeepWriteSYN framework overview showing the learning and inference phases for short-term and long-term handwriting structures.

The system utilizes two tunable parameters: temperature ($\tau$), which controls output randomness, and KL weight ($w_{KL}$), which dictates the degree of deviation from the input. This allows for the generation of diverse yet realistic variations of the original handwriting.

Experimental Setup

The model was evaluated using two datasets:

  • eBioDigitDB: On-line handwritten digits (0-9) from 93 subjects.
  • DeepSignDB (BiosecurID subset): A large-scale signature database featuring 1,526 subjects, including both genuine signatures and practiced "skilled forgeries."

Results and Discussion

DeepWriteSYN demonstrates significant improvements in both qualitative realism and quantitative performance. By modeling short-term strokes, the VAE learns the underlying "physics" of handwriting, allowing it to generalize variations to unseen subjects.

In one-shot signature verification tests, a baseline system using only one real signature achieved an Equal Error Rate (EER) of 19.79%. Augmenting this with 12 synthetic signatures generated by DeepWriteSYN reduced the EER to 9.36%. This performance represents a relative improvement of over 50% compared to the traditional "Sigma LogNormal" model, the previous industry standard.

Conclusion

DeepWriteSYN provides a powerful tool for data augmentation in handwriting biometrics. While highly effective at the stroke level, future work will address the challenges of modeling long-term dependencies during stroke concatenation and managing distortions at high variability settings. The researchers also plan to explore the synthesis of realistic forgeries to further enhance the security of biometric systems.

Summary updated

Abstract

This study proposes DeepWriteSYN, a novel on-line handwriting synthesis approach via deep short-term representations. It comprises two modules: i) an optional and interchangeable temporal segmentation, which divides the handwriting into short-time segments consisting of individual or multiple concatenated strokes; and ii) the on-line synthesis of those short-time handwriting segments, which is based on a sequence-to-sequence Variational Autoencoder (VAE). The main advantages of the proposed approach are that the synthesis is carried out in short-time segments (that can run from a character fraction to full characters) and that the VAE can be trained on a configurable handwriting dataset. These two properties give a lot of flexibility to our synthesiser, e.g., as shown in our experiments, DeepWriteSYN can generate realistic handwriting variations of a given handwritten structure corresponding to the natural variation within a given population or a given subject. These two cases are developed experimentally for individual digits and handwriting signatures, respectively, achieving in both cases remarkable results. Also, we provide experimental results for the task of on-line signature verification showing the high potential of DeepWriteSYN to improve significantly one-shot learning scenarios. To the best of our knowledge, this is the first synthesis approach capable of generating realistic on-line handwriting in the short term (including handwritten signatures) via deep learning. This can be very useful as a module toward long-term realistic handwriting generation either completely synthetic or as natural variation of given handwriting samples.
BibTeX
@inproceedings{tolosana2021deepwritesyn,
  author = {Ruben Tolosana and Paula Delgado-Santos and Andrés Pérez-Uribe and Rubén Vera-Rodríguez and Julian Fiérrez and Aythami Morales},
  title = {{DeepWriteSYN: On-Line Handwriting Synthesis via Deep Short-Term Representations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i1.16139},
}
Figure