Modern text prediction systems typically rely on the immediate prefix—the words a user has just typed—to suggest completions. However, human communication is inherently contextual. This research explores whether incorporating additional signals, such as conversation history, message subjects, and timestamps, can improve prediction accuracy in real-world applications like Microsoft Teams and Outlook. The study finds that while context significantly boosts performance in short-form chat, its impact on long-form email is more nuanced.
The Challenge of Contextual Prediction
Most text prediction engines are "prefix-based," guessing the next word based on the characters already present in the current message. Yet, human dialogue is reactive. A question like "Are you free for lunch?" makes a response starting with "I am free at..." highly probable. Researchers at Microsoft aimed to quantify the value of "extra" signals across different communication modalities, specifically comparing the rapid-fire nature of chat with the more formal, extended structure of email.
Methodology and System Architecture
To meet the strict low-latency requirements of a production service, the researchers utilized a two-layer Long Short-Term Memory (LSTM) network rather than a more computationally intensive Transformer model.
Contextual Encoding and Aggregation
The system employs a "joint encoding" strategy where contextual metadata—including time, subject lines, and message history—is concatenated into a single string. This string uses special tokens (e.g., <BOT> for "Beginning of Time") to separate metadata from the actual message prefix. For chat history, the study evaluated three aggregation modes: Ignore-Blocks (all sender messages in a window), Respect-Blocks (only the most recent continuous block), and Both-Senders (messages from both participants).
Datasets
The models were trained and evaluated on massive, anonymized production datasets:
- Microsoft Teams: 3.8 billion one-on-one chat messages.
- Microsoft Outlook: 150 million commercial emails.
- Avocado Dataset: A public email dataset used for qualitative sanity checks.
Key Insights: Why Chat and Email Differ
The most significant finding was the divergence between chat and email performance.
- The 5-Minute "Goldilocks" Zone for Chat: In Teams, a 5-minute history window proved optimal. Shorter windows often missed relevant context, while longer windows introduced irrelevant "noise." Crucially, the
Both-Sendersmode provided the strongest signal; knowing the recipient's question is vital for predicting the sender's answer. - Email Length as a Barrier: Email prediction benefited far less from conversation history. As shown in Figure 1, emails are significantly longer than chat messages (averaging 94 tokens vs. 9). In long-form text, the signal from previous messages is often overwhelmed by the current message's extensive prefix. The only consistent lift in email came from time-based signals, which improved predictions for temporal terms like "tomorrow" or "weekend."
Performance Results
The primary metric was Estimated Characters Accepted (Est. ChA), measuring the actual typing effort saved.
- Teams: The
Both-Senders5-minute window achieved a 9.3% to 18.6% relative gain over the non-contextual baseline. - Outlook: Contextual signals were less effective. Time context yielded a 2.88% gain, while subject lines and previous emails had negligible impact.
- Safety Improvements: Context also improved "blocklist" trigger rates. By analyzing the preceding conversation, the model could better suppress inappropriate suggestions in sensitive contexts.
Conclusion
The research highlights that chat and email require distinct AI strategies. While simple joint encoding works well for short chat messages, it may be a bottleneck for long-form emails. Future work could explore more sophisticated architectures, such as Hierarchical RNNs or Transformers, to better manage long-range dependencies in email threads.
Summary updated