OmniPrint: A Configurable Printed Character Synthesizer

Haozhe Sun, Wei-Wei Tu, Isabelle M Guyon

Introduction

Deep learning models for Optical Character Recognition (OCR) are notoriously data-hungry, yet high-quality labeled datasets are expensive and time-consuming to produce. While classic benchmarks like MNIST and Omniglot have driven progress, they possess inherent limitations: MNIST is increasingly considered too simple for modern architectures, and Omniglot’s hand-drawn strokes do not capture the specific geometric properties of printed text.

Furthermore, real-world OCR is often hindered by "nuisance variables"—environmental factors like lighting, motion blur, or unusual fonts that interfere with model performance. OmniPrint addresses these challenges by providing a highly configurable synthetic data generator capable of creating infinite variations of printed characters across 27 different scripts and nearly 1,000 fonts, offering a rigorous sandbox for benchmarking OCR, meta-learning, and domain adaptation.

Methods and Generative Pipeline

OmniPrint is built upon the Text Recognition Data Generator (TRDG) but introduces several critical technical upgrades to improve realism and control. The system follows a three-step probabilistic model: sampling nuisance variables (font, style, background), sampling a character label, and generating the final image.

The OmniPrint generative pipeline and transformation steps from vector representation to final textured image.

Pre-rasterization Transforms

The paper’s most significant technical contribution is the move to pre-rasterization transforms. Most synthesizers warp images after they have been turned into pixels (post-rasterization), which often introduces unnatural artifacts. OmniPrint uses the FreeType engine to manipulate the vector anchor points of a font before rendering. This allows for "natural" elastic distortions—such as stretching specific strokes—without losing image quality.

Seamless Blending

To simulate "in the wild" conditions, OmniPrint utilizes Poisson Image Editing. This technique ensures that when a character is pasted onto a natural background (such as stone or wood), the edges blend realistically rather than appearing as a digital overlay. The system is also modular, allowing for easy integration of new alphabets and custom transformation scripts.

Datasets and Coverage

The authors utilized OmniPrint to generate several benchmark sets for evaluation, most notably the OmniPrint-meta[1-5] series. These datasets range from simple black-on-white text (Meta-1) to complex images featuring sampled fonts, elastic transformations, and textured backgrounds (Meta-5).

Examples of synthetic characters generated by OmniPrint across various scripts and backgrounds, demonstrating the diversity of the dataset.

The initial release includes 12,729 characters from 27 scripts (including Arabic, Chinese, Devanagari, Greek, and Cyrillic) and 935 fonts. Additionally, the authors created "Fake-MNIST," a synthetic version of the MNIST dataset, to test how well models trained on synthetic data generalize to real-world handwritten digits.

Key Insights and Results

Vector Space vs. Pixel Space

By treating characters as geometric shapes rather than pixel grids, OmniPrint can imitate the variability of printing presses and handwriting more authentically. Interestingly, the authors found that "modeling difficulty" for AI does not always align with human intuition; models may find characters on textured backgrounds easier to recognize than humans do, provided the underlying mathematical features remain distinct.

Benchmarking and Domain Adaptation

  • Meta-Learning: Testing Prototypical Networks and MAML revealed a significant "robustness gap." While these models achieved ~98% accuracy on simple tasks, performance dropped to ~60-63% on the difficult OmniPrint-meta5 tasks.
  • Domain Adaptation: Using Fake-MNIST, the authors demonstrated that unsupervised domain adaptation methods could reach 98% accuracy on the real MNIST test set without ever seeing a real handwritten digit during training. This suggests that training on complex synthetic data is a viable strategy for solving real-world problems.
  • Regression Tasks: The system can also be used to train models to predict the specific amount of shear or rotation in an image, which has applications in forensic document analysis.

Limitations and Future Work

Currently, OmniPrint focuses on isolated characters. Future iterations aim to address full words and sentences, where kerning and ligatures become critical. The authors also plan to prioritize hyper-realistic effects such as complex shadows, sensor noise, and physical document degradation (e.g., folds or tears) to further bridge the gap between synthetic and real-world data.

Summary updated

Abstract

We introduce OmniPrint, a synthetic data generator of isolated printed characters, geared toward machine learning research. It draws inspiration from famous datasets such as MNIST, SVHN and Omniglot, but offers the capability of generating a wide variety of printed characters from various languages, fonts and styles, with customized distortions. We include 935 fonts from 27 scripts and many types of distortions. As a proof of concept, we show various use cases, including an example of meta-learning dataset designed for the upcoming MetaDL NeurIPS 2021 competition. OmniPrint is available at https://github.com/SunHaozhe/OmniPrint.
BibTeX
@misc{sun2022omniprint,
  author = {Haozhe Sun and Wei-Wei Tu and Isabelle M Guyon},
  title = {{OmniPrint: A Configurable Printed Character Synthesizer}},
  year = {2022},
  eprint = {2201.06648},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2201.06648},
}
Figure