Accelerating Text Communication via Abbreviated Sentence Input

Jiban Adhikary, Jamie Berger, Keith VertanenACL 2021

TL;DR

This paper introduces a novel approach to accelerate text communication by allowing users to abbreviate sentences, specifically by omitting mid-word vowels and spaces. They developed a robust recognition system that can expand these abbreviated inputs, achieving competitive speeds with traditional word prediction methods, particularly for users with motor impairments.

What Problem Does This Paper Address?

Typing on virtual keyboards, especially for users with motor disabilities (e.g., cerebral palsy, ALS), can be slow and inaccurate. While word completion helps, it can be cognitively demanding and challenging in eyes-free scenarios. This paper addresses the need for a more efficient and less cognitively burdensome text input method that can speed up communication without relying solely on traditional word prediction. They aim to make text entry faster and more accessible by enabling users to type less while still conveying their intended message accurately.

Methods

The core of this work lies in developing a system that can accurately expand abbreviated text. This involves several key components:

  • Abbreviation Strategy: Users are encouraged to omit mid-word vowels and spaces. This strategy was chosen based on previous work and a crowdsourced study that revealed these are common user behaviors.
  • Probabilistic Recognizer: An extension of the VelociTap touchscreen keyboard decoder is used. This decoder can handle noisy input (e.g., incorrect taps) and incorporates penalties for inserting characters. A novel addition is a specific penalty for inserting mid-word vowels, but only if the preceding character isn't a space, modeling the idea that vowels shouldn't be skipped at the start of words.
  • Neural Language Models (NLMs): NLMs are crucial for both training data selection and for improving the recognition accuracy.
  • Data Selection: To ensure the language models are well-suited for conversational text, they employ BERT-based classification to select relevant training data from a large corpus (Common Crawl). This is a departure from traditional n-gram based selection methods.
  • Rescoring: A Recurrent Neural Network Language Model (RNNLM) is used to rescore the n-best hypotheses generated by the decoder. This significantly improves the accuracy of the final recognized sentence.
  • Search Improvements: The decoder's search mechanism is modified to explicitly model the insertion of mid-word vowels.

Datasets

  • Training Data:
    • AAC-like crowdsourced messages (29K words) for in-domain data.
  • Common Crawl (1 billion words) for out-of-domain data, from which conversational sentences were selected using BERT-based classification.
  • Evaluation Data:
    • Enron mobile dataset (Vertanen and Kristensson, 2011a) and DailyDialog (Li et al., 2017) were used to evaluate the language models.
  • Touchscreen data collected from previous studies on phones and watches (Vertanen et al., 2015, 2013; Vertanen et al., 2018, 2019) was used for testing the recognition system. This data was adapted to simulate abbreviated input by dropping mid-word vowels.
  • A new crowdsourced study with 200 participants was conducted to observe abbreviation behavior.
  • A user study with 28 Amazon Mechanical Turk workers was performed to compare abbreviated sentence input with conventional word prediction.

Key Insights and Ideas

  • Vowel Omission is Key: The study found that omitting mid-word vowels is a highly effective strategy for users to abbreviate text, leading to significant compression without completely losing meaning.
  • Contextual Data Selection Matters: Using advanced models like BERT for selecting training data from large corpora significantly improves the performance of language models for conversational text, outperforming random selection and traditional cross-entropy methods.
  • NLMs are Powerful for Noisy Input: RNNLMs, when integrated into the decoding and rescoring process, dramatically improve the accuracy of recognizing abbreviated and noisy text, outperforming n-gram models.
  • User Behavior is Predictable (to an extent): Even without explicit instructions, users tend to omit vowels more than consonants and retain the first letter of words. However, some behaviors, like the high deletion rate of middle letters, were surprising.
  • Speed vs. Accuracy Trade-off: While abbreviated sentence input can achieve impressive accuracy, there's a trade-off with compression. Higher compression rates lead to higher error rates, but the system is designed to handle this noise.

Key Results

  • Compression: Abbreviating by dropping spaces and mid-word vowels can achieve sentence compression of 28% to 38%.
  • Recognition Accuracy: Even with 28% of letters omitted, sentences were recognized correctly 70% of the time in computational experiments. With a vowel drop probability of 0.5, the system achieved a character error rate (CER) of 4.4% when using RNNLM rescoring.
  • User Study Performance:
    • Entry Rate: Users achieved 9.6 words-per-minute (wpm) with abbreviated sentence input, compared to 9.9 wpm with conventional word prediction. This difference narrowed with practice.
  • Accuracy: The CER for abbreviated sentence input was 7.2%, compared to 0.3% for word input.
  • Correct Sentence Recognition: Users obtained a completely correct phrase 97% of the time in word mode and 68% in sentence mode.
  • Model Performance: BERT-selected training data resulted in the lowest perplexity and best recognition accuracy for character language models.

Limitations and Future Work

  • User Experience: The study acknowledges that users in the experiment might have had prior experience with mobile typing, potentially influencing their learning curve. Future work should validate the method with users who have actual rate-limited input abilities.
  • Abbreviation Strategies: The current model focuses on vowel omission and space removal. Future work could explore other abbreviation forms like phonetic replacement or consonant removal, and develop more sophisticated models for vowel insertion penalties.
  • Error Correction: While the system is accurate, errors still occur. Designing efficient and user-friendly interfaces for correcting these errors, especially for users with diverse motor abilities, is a key area for future research.
  • Data Collection: Obtaining large-scale, in-domain AAC-specific data for training is challenging due to privacy concerns. Exploring alternative data collection methods, such as audio-based input or having users compose novel abbreviated sentences, is suggested.
  • Generalization: The current system is trained on 100M characters. Scaling up training with larger language models (e.g., GPT-2) could further improve recognition accuracy.
  • Specific Word Handling: The system needs to better handle difficult words like acronyms or proper names, potentially by allowing users to specify words that should not be expanded or auto-corrected.

Summary updated

BibTeX
@inproceedings{adhikary2021accelerating,
  author = {Jiban Adhikary and Jamie Berger and Keith Vertanen},
  title = {{Accelerating Text Communication via Abbreviated Sentence Input}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure