Proactive Computation during Human Interaction Latency

This direction is merged into another direction: interactive query latency. Its last version stays readable and citable here.

Version 1 of 112 papers22 conceptsfrozen 2026-10-04

In traditional interactive systems, humans and machines operate in a strict "stop-and-wait" cycle. A user spends seconds or minutes typing a query, thinking about a visualization, or speaking a command; during this time, the computer sits idle. Once the user hits "Enter," the roles reverse: the computer works while the user waits.

This sequential turn-taking creates a massive temporal misalignment. Research across large language models (LLMs) and database systems suggests that human interaction latency—the "idle" time spent thinking, typing, or speaking—is not a bottleneck to be endured, but a computational resource to be harvested. By shifting from reactive to proactive execution, systems can effectively "hide" heavy computation within the natural pauses of human cognition.

The Hidden Resource: Think Time and Type Time

The window of opportunity for proactive computation is surprisingly large. In data science workflows, Doris Xin, Joseph M. Hellerstein, and their colleagues found that the 75th percentile of user "think time"—the interval between code executions—is roughly 23 seconds. Similarly, Philipp Eichmann and the creators of IDEBench: A Benchmark for Interactive Data Exploration noted that database systems often remain idle while a user is merely analyzing a visual output.

This "human latency" provides a window to perform three types of proactive work:

  1. Speculative Execution: Predicting and running likely future commands.
  2. Simultaneous Inference: Processing partial inputs as they arrive.
  3. State Management: Moving data to where it will be needed next.

Speculative Querying and Simultaneous Inference

The most direct way to use interaction time is to start the work before the user finishes the request. In the realm of LLMs, Chuangtao Chen and colleagues introduced LiveMind: Low-latency Large Language Models with Simultaneous Inference. Instead of waiting for a full prompt, LiveMind performs background reasoning on partial text or speech as it is being generated. By the time the user finishes their sentence, the model has already completed much of the "thinking," reducing perceived latency by up to 84%.

A similar logic applies to structured data. Srikanth Kandula and the team behind Speculative Ad-hoc Querying (SpeQL) use LLMs to predict the final SQL query while a user is still typing it. SpeQL doesn't just predict the text; it materializes "superset" temporary tables—results that are likely to contain the answer to the final query. When the user eventually submits the query, the system often only needs to perform a tiny refinement on precomputed data, turning tens of seconds of wait time into milliseconds.

Opportunistic Evaluation and Background Tasks

Not all proactive work requires predicting the future; some involves simply catching up on the past. In data science notebooks, many operations (like computing full descriptive statistics or logging) are not strictly necessary for the user's immediate next step.

Doris Xin et al. propose "opportunistic evaluation" in Enhancing the Interactivity of Dataframe Queries by Leveraging Think Time. Their system identifies non-critical tasks—which can account for over 50% of data science operations—and defers them. These tasks are then scheduled to run during the user's "think time." This ensures the system stays responsive for the "interaction-critical path" while still completing the full workload in the background.

Proactive State Migration

In distributed systems, the bottleneck isn't always computation, but memory. For multi-turn AI interactions, the "KV cache" (the model's memory of the conversation) must be available on the specific GPU performing the inference.

Saurabh Agarwal and the authors of SYMPHONY: Improving Memory Management for LLM Inference Workloads leverage the inherent delays in human-AI dialogue to solve this. SYMPHONY uses "advisory requests"—predictive hints that a user is likely to respond—to proactively migrate and prefetch these memory states across a cluster. By moving the data while the user is still reading the previous response or typing a new one, the system eliminates the transfer delay that usually plagues large-context models.

Why It Works: The Asymmetry of Speed

The success of these methods relies on a fundamental asymmetry: computers are orders of magnitude faster than human interaction, but humans are the ultimate bottleneck for "submitting" tasks. By treating the human interaction phase as a slow-motion stream of information rather than a single trigger event, these systems transform idle time into a buffer. Whether it is pre-calculating SQL joins in SpeQL or migrating caches in SYMPHONY, the goal is the same: to ensure that when the human is finally ready for the machine, the machine has already finished the work.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_proactive_computation_during_human_inter_ca1f493f,
  title = {Proactive Computation during Human Interaction Latency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/proactive-computation-during-human-interaction-latency/rdr_ca1f493f81fc46939bb5afac5848f811},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_proactive_computation_during_human_inter_ca1f493f_v1,
  title = {Proactive Computation during Human Interaction Latency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/proactive-computation-during-human-interaction-latency/rdr_ca1f493f81fc46939bb5afac5848f811/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}