Abstract
Zezhong Jin, Shubhang Desai, Xu Chen, Biyi Fang, Zhuoyi Huang, Zhe Li, Chong-Xin Gan, Xiao Tu, Man-Wai Mak, Yan Lu, Shujie Liu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.
2025.
Introduction
Handwriting synthesis is categorized into offline (static images) and online (stroke-based sequences) generation. Online synthesis, or "ink generation," produces lightweight vector data representing $(x, y)$ coordinates and pen states, capturing the temporal dynamics of writing. While Recurrent Neural Networks (RNNs) like LSTMs have long been the standard, they suffer from sequential processing bottlenecks and difficulties in modeling long-range dependencies.
Transformers offer a powerful alternative but face a significant challenge in ink generation: alignment. Unlike text-to-text tasks, a single character corresponds to a variable, much larger number of stroke points. Without the inherent sequential bias of RNNs, standard Transformers struggle to maintain the monotonic flow of handwriting. This paper introduces TrInk, a Transformer-based architecture designed to overcome these hurdles.
Methodology
TrInk utilizes an encoder-decoder Transformer framework specifically adapted for handwriting dynamics.
Gaussian Memory Mask
The core innovation is the Gaussian memory mask applied to the cross-attention module. This mask enforces a monotonic alignment by focusing the decoder on a shifting "window" of the input text. This inductive bias ensures that the model generates strokes in the correct character order, preventing the chaotic alignment often seen in vanilla Transformers for this task.
Scaled Positional Encoding
To account for the differing scales and characteristics of text tokens versus stroke points, TrInk employs scaled positional encodings. Standard sinusoidal embeddings are modified with a learnable scale factor ($\alpha$), allowing the model to adaptively weight positional information for the encoder and decoder independently.
Mixture Density Network (MDN)
The decoder outputs parameters for a Gaussian Mixture Model via an MDN. This approach captures the stochastic nature of human handwriting, allowing for realistic variations in stroke placement and "wiggle."
Experimental Results
The model was trained on a curated dataset of 600,000 high-quality ink samples and evaluated on the IAM-OnDB benchmark.
Quantitative Performance
TrInk achieved a 35.56% reduction in Character Error Rate (CER) and a 29.66% reduction in Word Error Rate (WER) compared to previous state-of-the-art RNN models. The improvement was most pronounced in long text sequences (>40 characters), where CER was reduced by 56.41%, demonstrating the Transformer's superior ability to handle long-range dependencies when constrained by a monotonic mask.
Subjective Evaluation
Human raters evaluated the generated ink based on legibility and stylistic consistency. TrInk consistently outperformed baselines like AlexRNN across both metrics.
Conclusion
TrInk represents a significant advancement in online handwriting synthesis by successfully adapting the Transformer architecture to a task requiring strict monotonic alignment. While computationally more intensive than RNNs, its superior performance on long-form text and higher subjective quality scores establish a new benchmark for the field. Future work will focus on expanding the model to support multilingual scripts and optimizing it for mobile deployment.
Summary updated