Handwriting Prediction Considering Inter-Class Bifurcation Structures

Masaki Yamagata, Hideaki Hayashi, Seiichi Uchida

Introduction

Predicting the future trajectory of handwriting is a complex task in time-series analysis. Unlike simple physical systems, handwriting is driven by human intent, which is often ambiguous in its early stages. A significant challenge in this domain is inter-class bifurcation: the phenomenon where different characters (e.g., '0' and '6') share nearly identical initial strokes. Standard predictive models often fail here, either by averaging the possible paths into a nonsensical "middle ground" or by prematurely committing to a single, potentially incorrect, class.

The Class-Guided Prediction (CGP) Model

To address this, the authors propose the Class-Guided Prediction (CGP) model. The core innovation lies in factorizing the prediction task into two distinct components: class probability and class-conditional coordinate prediction.

Figure 1: The CGP architecture uses a three-layer LSTM with skip connections to feed dual output modules for class probability and class-specific GMM parameters.

As shown in Figure 1, the model extracts features from a sequence of relative pen coordinates using a 3-layer LSTM with skip connections. These features are then processed by two parallel modules:

  1. Class Probability Prediction: Outputs a categorical distribution representing the likelihood of each digit class (0-9).
  2. Coordinate Prediction: Outputs parameters for a Gaussian Mixture Model (GMM) for each class, defining the specific trajectory dynamics for that character.

By training these components jointly using a combined loss function (negative log-likelihood and cross-entropy), the model learns to represent multiple distinct futures simultaneously. This prevents "mode collapse," where a model merges diverging paths into a single, high-variance blob.

Handling Ambiguity and Bifurcation

The CGP model treats uncertainty as a structural feature rather than a bug. Early in a stroke, when the writer's intent is unclear, the class probabilities are distributed across several candidates. Consequently, the sampled future trajectories branch out, representing all likely characters. As the pen moves and the ambiguity is resolved, the class probabilities shift toward the true character, and the predicted trajectories naturally collapse into a single, accurate path.

Figure 5: Evolution of predicted trajectories over time. As more of the stroke is observed, the model narrows its hypotheses, eventually converging on the correct character path.

Quantitative analysis using the UNIPEN database confirms this behavior. The authors observed that prediction error (RMSE) tends to spike at bifurcation points—where paths for different digits diverge—before dropping once the class becomes identifiable. In comparative tests, CGP significantly outperformed deterministic LSTMs and standard MDNs in long-term prediction accuracy, particularly because it maintains distinct modes for different classes rather than blurring them.

Conclusion

The CGP framework provides a robust method for handwriting prediction by explicitly modeling the underlying class structure. By separating "what is being written" from "how it is written," the model handles inter-class bifurcations gracefully. Future work may extend this to continuous handwriting or leverage the bifurcation detection for ultra-fast character recognition, identifying a digit the moment its trajectory becomes unique.

Summary updated

BibTeX
@inproceedings{yamagata2020handwriting,
  author = {Masaki Yamagata and Hideaki Hayashi and Seiichi Uchida},
  title = {{Handwriting Prediction Considering Inter-Class Bifurcation Structures}},
  booktitle = {ICFHR},
  year = {2020},
  doi = {10.1109/icfhr2020.2020.00029},
}
Figure