Inkitiate: Initiating and Steering Agent Assistance through Ink in a Spatial Workspace

Liangwei Wang, Nan Chen, Luna K. Qiu, Zilong Wang, Fugee Tsung, Yuyu Luo, Yuqing YangarXiv 2026 · arXiv

TL;DR

Inkitiate is a spatial knowledge workspace that allows users to initiate and steer AI agent assistance using digital ink—handwriting and sketching—directly on a canvas. By combining linguistic requests with spatial and graphical marks, users can retrieve, analyze, and organize information in a persistent environment where AI results become immediate building blocks for further thought.

What Problem Does This Paper Address?

Knowledge work is rarely a straight line; it’s a "sensemaking" process of gathering, interpreting, and organizing information. While Large Language Models (LLMs) are powerful assistants, current interfaces often force a "chat-box" paradigm that is disconnected from the spatial way people naturally think (e.g., sketching on whiteboards or annotating margins).

Existing AI tools often lack context about the user's current workspace and provide "terminal" results—static text blocks that are hard to reuse. The authors identify a need for a "pen-first" interaction model where ink acts as both content and instruction, allowing users to steer agents through the same medium they use to externalize their own ideas.

Methods

The researchers developed Inkitiate, a web-based spatial workspace optimized for stylus-enabled devices. The system's design was informed by a formative study with eight participants and is built on four core design goals:

  1. Task Specification: Combining ink with existing canvas materials to define tasks.
  2. Bounded Context: Carrying forward relevant history without cluttering the agent's view with the entire canvas.
  3. Inspectability: Keeping agent execution transparent and interruptible.
  4. Persistent Results: Writing results back as editable, reusable canvas objects.
Figure 1: Inkitiate is a spatial knowledge workspace where users interact with diverse canvas objects and use ink expressions to initiate and steer agent assistance directly on the canvas. Here, Amy explores Odysseus’s journey from Troy to Ithaca by working with web resources, text, drawings, and spatial paths. She (1) creates a bounded query from a selected ink expression, (2) inspects and steers the agent’s execution, and (3) incorporates its organized, reusable results into her evolving canvas.

Key technical features include:

  • Bounded Task Input: Users "lasso" ink and objects to define the scope of a query.
  • Handwritten Sigils: Special marks like # (to bind resources) and // (to specify agent capabilities like "Find Resource") provide explicit control.
  • Agent Activity Cards: These cards appear on the canvas to show the agent's interpreted intent, current progress, and elapsed time, allowing users to stop or steer the task mid-execution.
  • Architecture: A host server mediates between the persistent canvas and a general-purpose agent runtime (using GPT-5.6-Sol), ensuring that AI actions are bounded and results are structured before being written back.

Datasets

As a system-focused paper, Inkitiate was not trained on a new dataset. Instead, the authors performed a Scenario-Based System Validation using 34 distinct ink-query cases to ensure stability and correctness across various combinations of ink and object types. The primary evaluation was a user study with 18 participants who performed both prescribed tasks (Reproduction) and open-ended exploration (Freeform Creation).

Key Insights and Ideas

  • Ink is Multimodal: The study identified five distinct roles for ink: Linguistic (writing), Depictive (sketching), Referential (pointing to objects), Relational (connecting ideas), and Spatial (layout cues). Users rarely used just one; they typically combined them to create expressive, compact requests.
  • Prior Outputs are Materials: In the freeform study, 69.1% of queries reused previously generated results. This confirms that in a spatial workspace, AI results are not the end of a conversation but the beginning of the next step.
  • Bounded Continuity: Users don't want the agent to see the whole canvas, but they do want it to remember the "thread" of the current task. Inkitiate achieves this by defining a "neighborhood" of context based on the location of the ink query.
  • The Value of Transparency: Participants highly valued the ability to see how the agent interpreted their messy handwriting before it finished a long task, allowing them to intervene early if the agent went off-track.
Figure 3: A bounded task input in Inkitiate. In this example, the user lassos and explicitly submits free-form ink together with selected canvas objects and an optional sigil. Nearby materials outside the submission boundary remain on the persistent canvas but are not automatically included as task inputs.

Key Results

  • High Usability: The system received an average System Usability Scale (SUS) score of 5.93 out of 7, with particularly high marks for learnability (6.33) and user confidence (6.28).
  • Low Workload: NASA-TLX ratings were consistently below the scale midpoint, suggesting that steering an agent with ink is not mentally or physically taxing.
  • Complex Structuring: Users successfully used the agent to transform raw information into complex visual structures like timelines, branching trees, and radial constellations.
  • Steerability: The "Agent Activity Card" proved effective; for example, one user stopped an agent that was trying to generate a map and redirected it to retrieve a real-world map using the //Find Resource sigil.
Figure 4: Inspecting and intervening in an agent task. (1) After the user submits a request for an illustrated map of Odysseus’s journey, an agent activity card displays the understood intent and current activity at the location of the ink query. (2) Expanding the card reveals the activity history, current step, elapsed time, and a control for stopping the task. (3) When the resource search returns multiple candidates, the user selects which maps to import into the canvas.

Limitations and Future Work

  • Fine-Grained Targeting: While the system handles whole objects well, it struggles when users point to specific words inside a text block or specific regions inside an image.
  • Interpretation Uncertainty: Free-form ink is inherently ambiguous. While the clarification interface helps, future versions could benefit from better handwriting and vision models to reduce the need for manual correction.
  • Long-term Use: The study was limited to single sessions. The authors suggest that longitudinal studies are needed to see how personal "ink vocabularies" evolve over weeks or months of daily work.
  • Multimodal Integration: Participants expressed interest in combining ink with speech for longer, more detailed instructions, suggesting a path for future "sketch-and-speech" interfaces.

Summary updated

Abstract

Handwriting and sketching help people develop ideas around information. As this work unfolds, users may seek assistance with finding, understanding, and organizing materials. We investigate how ink can initiate and steer agent assistance in this process. An eight-participant formative study explored desired assistance and interaction expectations, informing three connected activities: bringing materials in, thinking with materials, and shaping the space. We present Inkitiate, a spatial workspace for ink-based agent assistance in knowledge work. Through ink, users retrieve external materials, build on accumulated content and results, and reorganize the canvas, while inspecting and guiding agent actions. In a study with eighteen participants, users reported favorable usability and low perceived workload, and used retained ink and materials to develop further ink queries. An ink-query vocabulary and cases reveal how users combine linguistic and graphical expressions to direct agents. These findings inform ink-based agent interaction design for expressive requests and continued work with materials.
BibTeX
@misc{wang2026inkitiate,
  author = {Liangwei Wang and Nan Chen and Luna K. Qiu and Zilong Wang and Fugee Tsung and Yuyu Luo and Yuqing Yang},
  title = {{Inkitiate: Initiating and Steering Agent Assistance through Ink in a Spatial Workspace}},
  year = {2026},
  eprint = {2610.39905},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.39905},
}
Figure