TrInk: Ink Generation with Transformer Network

Zezhong Jin, Sujay B. Desai, Xin Chen, Biyi Fang, Zhiqiang Huang, Zhe Li, Chong-Xin Gan, Xiu-Ping Tu, Man‐Wai Mak, Yan Lü, Shujie Liu2025 · DOI

Abstract

Zezhong Jin, Shubhang Desai, Xu Chen, Biyi Fang, Zhuoyi Huang, Zhe Li, Chong-Xin Gan, Xiao Tu, Man-Wai Mak, Yan Lu, Shujie Liu. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

2025.

Introduction

Handwriting synthesis is categorized into offline (static images) and online (stroke-based sequences) generation. Online synthesis, or "ink generation," produces lightweight vector data representing $(x, y)$ coordinates and pen states, capturing the temporal dynamics of writing. While Recurrent Neural Networks (RNNs) like LSTMs have long been the standard, they suffer from sequential processing bottlenecks and difficulties in modeling long-range dependencies.

Transformers offer a powerful alternative but face a significant challenge in ink generation: alignment. Unlike text-to-text tasks, a single character corresponds to a variable, much larger number of stroke points. Without the inherent sequential bias of RNNs, standard Transformers struggle to maintain the monotonic flow of handwriting. This paper introduces TrInk, a Transformer-based architecture designed to overcome these hurdles.

Methodology

TrInk utilizes an encoder-decoder Transformer framework specifically adapted for handwriting dynamics.

The TrInk architecture featuring an encoder for text and a decoder for stroke sequences, connected via monotonic cross-attention.

Gaussian Memory Mask

The core innovation is the Gaussian memory mask applied to the cross-attention module. This mask enforces a monotonic alignment by focusing the decoder on a shifting "window" of the input text. This inductive bias ensures that the model generates strokes in the correct character order, preventing the chaotic alignment often seen in vanilla Transformers for this task.

Scaled Positional Encoding

To account for the differing scales and characteristics of text tokens versus stroke points, TrInk employs scaled positional encodings. Standard sinusoidal embeddings are modified with a learnable scale factor ($\alpha$), allowing the model to adaptively weight positional information for the encoder and decoder independently.

Mixture Density Network (MDN)

The decoder outputs parameters for a Gaussian Mixture Model via an MDN. This approach captures the stochastic nature of human handwriting, allowing for realistic variations in stroke placement and "wiggle."

Experimental Results

The model was trained on a curated dataset of 600,000 high-quality ink samples and evaluated on the IAM-OnDB benchmark.

Quantitative Performance

TrInk achieved a 35.56% reduction in Character Error Rate (CER) and a 29.66% reduction in Word Error Rate (WER) compared to previous state-of-the-art RNN models. The improvement was most pronounced in long text sequences (>40 characters), where CER was reduced by 56.41%, demonstrating the Transformer's superior ability to handle long-range dependencies when constrained by a monotonic mask.

Subjective Evaluation

Human raters evaluated the generated ink based on legibility and stylistic consistency. TrInk consistently outperformed baselines like AlexRNN across both metrics.

Human evaluation results showing TrInk's superiority in legibility and style consistency compared to RNN-based methods.

Conclusion

TrInk represents a significant advancement in online handwriting synthesis by successfully adapting the Transformer architecture to a task requiring strict monotonic alignment. While computationally more intensive than RNNs, its superior performance on long-form text and higher subjective quality scores establish a new benchmark for the field. Future work will focus on expanding the model to support multilingual scripts and optimizing it for mobile deployment.

Summary updated

BibTeX
@misc{jin2025trink,
  author = {Zezhong Jin and Sujay B. Desai and Xin Chen and Biyi Fang and Zhiqiang Huang and Zhe Li and Chong-Xin Gan and Xiu-Ping Tu and Man‐Wai Mak and Yan Lü and Shujie Liu},
  title = {{TrInk: Ink Generation with Transformer Network}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.244},
}
Figure