Proofread: Fixes All Errors with One Tap

Renjie Liu, Yanxiang Zhang, Yun Zhu, Haicheng Sun, Yuanbo Zhang, Michael Huang, Shanqing Cai, Lei Meng, Shumin ZhaiACL 2024

TL;DR

This paper introduces "Proofread," a new feature for the Gboard mobile keyboard that leverages large language models (LLMs) to fix sentence- and paragraph-level errors with a single tap. The system involves a novel data synthesis pipeline, a two-stage model tuning process (SFT and RL), and efficient deployment techniques to deliver high-quality corrections with low latency.

What Problem Does This Paper Address?

Typing on mobile devices, especially with "fat finger" syndrome on small screens, is inherently error-prone, with per-letter error rates around 8-9%. While Gboard offers various correction features (active key correction, auto-correction, word completion, spell checker, grammar checker), these have limitations. Active corrections struggle with complex, longer-span errors, and passive features require users to manually engage in multi-step correction processes. This leads to a trade-off between typing speed and accuracy, forcing users to either type slowly and carefully or spend significant cognitive effort correcting errors post-typing. The paper aims to address this by providing a seamless, one-tap solution for comprehensive error correction.

Methods

The Proofread system is built around a carefully tuned LLM, specifically a variant of PaLM2-XS. The methodology involves several key stages:

  • Data Generation: A synthetic error framework is used to generate data that mimics common typing errors on Gboard. This is followed by a Gboard simulator to fix errors using existing decoding, key correction, and auto-correction functions. Heuristic rules are applied for specific cases like emoji alignment and URL formatting. Finally, LLM-based filtering ensures data quality.
  • Metrics Design: A suite of metrics is developed to evaluate model quality from various dimensions, including exact match (EM), normalized exact match (NEM), error ratio, different meaning ratio, good ratio, and bad ratio. The "Good Ratio" and "Bad Ratio" are highlighted as primary metrics, combining grammar error and meaning preservation checks.
  • Model Tuning: A two-stage tuning process is employed:
  • Supervised Fine-Tuning (SFT): The model is first fine-tuned on a rewrite dataset and then on the synthetically generated proofreading dataset. Sequential tuning on rewrite and proofread tasks proved most effective.
  • Reinforcement Learning (RL) Tuning: Inspired by InstructGPT, Reinforcement Learning from AI Feedback (RLAIF) is used with heuristic rewards. Two novel reward strategies are introduced: "Global Reward" (LLM judges candidate quality based on few-shot examples) and "Direct Reward" (directly converting grammar and meaning checks into rewards). Proximal Policy Optimization (PPO) is used for optimization, with KL divergence to maintain original text recovery.
  • Latency Optimization: To ensure real-time performance on mobile devices, several techniques are applied:
  • Quantization: 8-bit quantization is used to reduce memory footprint and latency.
  • Bucket Keys: Input data is categorized into discrete buckets based on sentence length.
  • Text Segmentation: Documents are segmented into paragraphs for parallel processing.
  • Speculative Decoding: This technique leverages hardware parallelism to pre-conduct computation with a sampling method, significantly reducing inference speed.

Datasets

  • Web Crawled Dataset: Used as the initial source for data generation.
  • Gboard Simulator: Used to fix errors in the synthetically generated data.
  • Rewrite Dataset: A collection of text rewriting tasks from existing works (Shu et al., Zhu et al.).
  • Synthetic Proofread Dataset: Generated using the described synthetic error framework.
  • Golden Set: A human-labeled dataset for evaluation.

Key Insights and Ideas

  • LLMs for Comprehensive Correction: The paper demonstrates that LLMs can effectively handle sentence- and paragraph-level error correction, going beyond the capabilities of traditional on-device models.
  • Synthetic Data Quality: A carefully designed synthetic data generation pipeline is crucial for training LLMs to mimic real-world typing errors and achieve high performance.
  • Two-Stage Tuning is Key: The combination of SFT (especially with sequential rewrite and proofread tuning) and RL tuning is vital for achieving both foundational quality and targeted refinement.
  • Novel RL Rewards: The "Global Reward" and "Direct Reward" strategies offer effective ways to guide LLMs towards better proofreading without extensive human labeling.
  • Speculative Decoding for Latency: This technique is a significant enabler for deploying LLMs in latency-sensitive applications like mobile keyboards.

Key Results

  • The tuned PaLM2-XS model achieved an impressive 85.56% Good Ratio and 14.44% Bad Ratio on a golden set.
  • RL tuning with Global Reward improved the Bad Ratio by 3.65%, and with Direct Reward by 5.74% (relative improvements).
  • Speculative decoding reduced median latency by 39.4%.
  • The Proofread feature has been launched to thousands of daily active users on Pixel 8 devices.

Limitations and Future Work

  • Metric Conflicts: The authors suspect inherent conflicts in their defined metrics, as optimizing for grammar correction (reducing error ratio) sometimes negatively impacts exact match ratios.
  • Meaning Subjectivity: Optimizing for meaning alignment is noted as more subjective and complex than grammar correction.
  • Future Work:
    • Leveraging real-user data for further improvement.
  • Multilingual adaptation of the Proofread feature.
  • Personalized assistance for diverse writing styles.
  • Exploring privacy-preserving on-device solutions.

Summary updated

BibTeX
@inproceedings{liu2024proofread,
  author = {Renjie Liu and Yanxiang Zhang and Yun Zhu and Haicheng Sun and Yuanbo Zhang and Michael Huang and Shanqing Cai and Lei Meng and Shumin Zhai},
  title = {{Proofread: Fixes All Errors with One Tap}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
}
Figure