tok: a textual concrete syntax of BPMN and DMN for humans and AI agents

Marek DejaarXiv 2026 · arXiv

TL;DR

tok is a novel plain-text notation for business process (BPMN) and decision (DMN) models designed to be equally readable by humans and AI agents. By stripping away the verbosity of standard XML, it reduces model size by up to 7.7x while providing a powerful Rust-based engine that outperforms traditional validators in catching modeling errors.

What Problem Does This Paper Address?

In the world of enterprise software, Business Process Model and Notation (BPMN) and Decision Model and Notation (DMN) are the gold standards for documenting how work gets done. However, their primary exchange format is XML—a format that is notoriously difficult for humans to read, nearly impossible to "diff" in version control, and incredibly "expensive" for Large Language Models (LLMs) due to its high token count.

Currently, developers and analysts face a "three-reader" problem:

  1. Humans need visual diagrams.
  2. Process engines need precise XML.
  3. AI agents need concise, structured text.

Existing textual notations often compromise, either losing the semantic richness of the standards to gain simplicity (like Mermaid) or remaining too tied to the underlying XML structure. The author introduces tok to bridge this gap, providing a lossless, token-efficient textual syntax that serves all three audiences simultaneously.

Methods

The author developed tok as a line-oriented, indentation-based notation (following the "offside rule" similar to Python). It maps one-to-one to the BPMN and DMN metamodels, ensuring that any model written in tok can be compiled into standard-compliant XML or rendered as a diagram.

Key technical components include:

  • The Engine: A high-performance library written in Rust that handles parsing, validation, and compilation.
  • Automatic Layout: Unlike many tools that require manual coordinate entry, the tok engine automatically computes the visual layout for diagrams (SVG).
  • Layered Diagnostics: The checker provides localized error messages (line and element specific), which act as a "grounding channel" for both human editors and AI agents.
  • Model Context Protocol (MCP): The engine is exposed via an MCP server, allowing AI agents to use tok tools (like check or fmt) directly during a conversation to self-correct their output.
The tok engine architecture allows it to serve various interfaces from a single Rust library.

Datasets

The study evaluated tok using several diverse corpora:

  • MIWG Test Suite: 893 BPMN files exported by 37 different industry tools to test "real-world" compatibility and token reduction.
  • DMN TCK: 143 models from the Decision Model and Notation Technology Compatibility Kit.
  • PET & PMo: Two datasets containing 87 natural language process descriptions used to test the ability of LLMs (Claude 3.5, GPT-4 class models) to generate models from scratch.
  • Corpus N: A collection of 93 hand-crafted tok models used for performance and fault-injection benchmarks.

Key Insights and Ideas

  • Separation of Logic and Geometry: By moving the "analog" part of a drawing (coordinates, bend points) into a separate layout file or letting the engine compute it, the tok text remains purely focused on process logic. This is the primary driver of its massive token savings.
  • The "Checker" as a Teacher: The paper demonstrates that an LLM doesn't need to be pre-trained on a new notation if it has access to a high-quality checker. By providing a one-page reference and a tool that points out specific errors, even models "new" to tok can write valid code with near-perfect accuracy.
  • Token Efficiency = Lower Cost: For AI agents, tokens are money. Reducing a model's size by 2-4x directly reduces the cost of using LLMs for automated process discovery and modeling.

Key Results

The quantitative benefits of tok are significant:

  • Efficiency: A tok model uses 2.2x fewer tokens than the smallest minified BPMN XML and 7.7x fewer than typical files exported by modeling tools.
  • Robustness: In a fault-injection study of 1,411 errors, the tok checker signaled 98.2% of faults, compared to only 85.8% for standard XML Schema and bpmnlint combinations.
  • AI Capability: Using a "lean skill" (a simple 5-step instruction set and a checker tool), LLMs like Claude 3.5 Sonnet wrote valid tok for 100% of the test descriptions, spending significantly fewer tokens than they would have on standard XML.
Figure 1: The diagram that tok svg draws from the interlibrary-loan document of Listing 1, with the automatic layout of the engine, showing one pool, two lanes, an exclusive gateway with two labelled outcomes and two end events.

Limitations and Future Work

While tok is highly effective for structural and semantic validation, the author notes some limitations:

  • Behavioral Analysis: The current checker does not perform deep state-space exploration to find complex behavioral deadlocks (e.g., specific parallel join conflicts).
  • Human Factors: While the notation is designed for humans, the study focused primarily on LLM and tool benchmarks. A formal user study is needed to quantify how much faster humans can read or debug tok compared to XML or graphical tools.
  • Tool Ecosystem: While it supports MCP and LSP, wider adoption will depend on integration into existing enterprise BPMN suites.

The author suggests that future work could involve "grammar-constrained decoding," where the LLM's output is forced to follow the tok syntax at generation time, potentially further increasing reliability.

Summary updated

Abstract

tok is a plain-text notation for Business Process Model and Notation (BPMN) 2.0.2 process models and Decision Model and Notation (DMN) 1.5 decisions. One engine checks a tok document against both standards, compiles the document to XML with an automatic diagram, and serves language-model applications through the Model Context Protocol. The paper classifies the notation as an alternative concrete syntax of the BPMN and DMN metamodels and reports one study. A tok model takes 2.2 times fewer tokens than the smallest BPMN XML of 893 files exported by 37 tools and 7.7 times fewer than the files as exported. The checker signals 98.2% of 1,411 injected faults, against 85.8% for the XML Schema with bpmnlint. Four language models new to tok write valid tok from a one-page reference for 85.7 to 100% of 87 process descriptions from two datasets and valid BPMN XML for every description, and the strongest model writes tok with 25% and 19% fewer tokens than BPMN XML. As agents with a lean skill and one checker tool, the models write valid tok for 97.6 to 100% of the descriptions, the strongest model spends 21% and 14% fewer tokens on tok than on BPMN XML, and a skill which sends the model to read the reference through a tool costs 1.7 to 2.8 times the tokens of the lean skill. Of 47 event logs simulated from public models, 44 fit their source models, and tok imports every model discovered from the logs.
BibTeX
@misc{deja2026tok,
  author = {Marek Deja},
  title = {{tok: a textual concrete syntax of BPMN and DMN for humans and AI agents}},
  year = {2026},
  eprint = {2611.07945},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.07945},
}
Figure