When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages

Stojan Trajanovski, Chad Atalla, Kunho Kim, Vipul Agarwal, Milad Shokouhi, Chris Quirk

Modern text prediction systems typically rely on the immediate prefix—the words a user has just typed—to suggest completions. However, human communication is inherently contextual. This research explores whether incorporating additional signals, such as conversation history, message subjects, and timestamps, can improve prediction accuracy in real-world applications like Microsoft Teams and Outlook. The study finds that while context significantly boosts performance in short-form chat, its impact on long-form email is more nuanced.

The Challenge of Contextual Prediction

Most text prediction engines are "prefix-based," guessing the next word based on the characters already present in the current message. Yet, human dialogue is reactive. A question like "Are you free for lunch?" makes a response starting with "I am free at..." highly probable. Researchers at Microsoft aimed to quantify the value of "extra" signals across different communication modalities, specifically comparing the rapid-fire nature of chat with the more formal, extended structure of email.

Methodology and System Architecture

To meet the strict low-latency requirements of a production service, the researchers utilized a two-layer Long Short-Term Memory (LSTM) network rather than a more computationally intensive Transformer model.

Contextual Encoding and Aggregation

The system employs a "joint encoding" strategy where contextual metadata—including time, subject lines, and message history—is concatenated into a single string. This string uses special tokens (e.g., <BOT> for "Beginning of Time") to separate metadata from the actual message prefix. For chat history, the study evaluated three aggregation modes: Ignore-Blocks (all sender messages in a window), Respect-Blocks (only the most recent continuous block), and Both-Senders (messages from both participants).

Figure 3: Overview of the input data representation and the three chat history aggregation modes.

Datasets

The models were trained and evaluated on massive, anonymized production datasets:

  • Microsoft Teams: 3.8 billion one-on-one chat messages.
  • Microsoft Outlook: 150 million commercial emails.
  • Avocado Dataset: A public email dataset used for qualitative sanity checks.

Key Insights: Why Chat and Email Differ

The most significant finding was the divergence between chat and email performance.

  • The 5-Minute "Goldilocks" Zone for Chat: In Teams, a 5-minute history window proved optimal. Shorter windows often missed relevant context, while longer windows introduced irrelevant "noise." Crucially, the Both-Senders mode provided the strongest signal; knowing the recipient's question is vital for predicting the sender's answer.
  • Email Length as a Barrier: Email prediction benefited far less from conversation history. As shown in Figure 1, emails are significantly longer than chat messages (averaging 94 tokens vs. 9). In long-form text, the signal from previous messages is often overwhelmed by the current message's extensive prefix. The only consistent lift in email came from time-based signals, which improved predictions for temporal terms like "tomorrow" or "weekend."
Figure 1: Distribution of tokens per message for chat and email, illustrating the significant difference in message length.

Performance Results

The primary metric was Estimated Characters Accepted (Est. ChA), measuring the actual typing effort saved.

  • Teams: The Both-Senders 5-minute window achieved a 9.3% to 18.6% relative gain over the non-contextual baseline.
  • Outlook: Contextual signals were less effective. Time context yielded a 2.88% gain, while subject lines and previous emails had negligible impact.
  • Safety Improvements: Context also improved "blocklist" trigger rates. By analyzing the preceding conversation, the model could better suppress inappropriate suggestions in sensitive contexts.

Conclusion

The research highlights that chat and email require distinct AI strategies. While simple joint encoding works well for short chat messages, it may be a bottleneck for long-form emails. Future work could explore more sophisticated architectures, such as Hierarchical RNNs or Transformers, to better manage long-range dependencies in email threads.

Summary updated

BibTeX
@inproceedings{trajanovski2021when,
  author = {Stojan Trajanovski and Chad Atalla and Kunho Kim and Vipul Agarwal and Milad Shokouhi and Chris Quirk},
  title = {{When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-industry.1},
}
Figure