TD-Interpreter: Enhancing the Understanding of Timing Diagrams with Visual-Language Learning

Jie He, Vincent Theo Willem Kenbeek, Zhantao Yang, Meixun Qu, Ezio Bartocci, Dejan Nickovic, Radu GrosuarXiv 2025 · DOI · arXiv

Introduction

In digital design, timing diagrams (TDs) serve as the fundamental language for describing signal interactions over time. For engineers, deciphering these diagrams—whether from third-party datasheets or complex simulation traces—is often a tedious and error-prone process. While general-purpose Large Language Models (LLMs) like GPT-4o excel in natural language tasks, they frequently struggle with the precise technical reasoning required to interpret hardware waveforms, often leading to "hallucinations."

To bridge this gap, researchers from TU Wien and their collaborators have developed TD-Interpreter, a specialized vision-language model (VLM) designed to assist in digital logic verification. By fine-tuning a lightweight model on a custom synthetic dataset, the team created a tool capable of interpreting waveforms with a level of precision that surpasses state-of-the-art commercial models.

The Challenge of Automated Waveform Interpretation

Interpreting timing diagrams presents unique challenges for standard AI models:

  1. Domain Specificity: General VLMs are typically trained on natural images rather than technical hardware protocols.
  2. Data Scarcity: There is a lack of large-scale, open-source datasets pairing timing diagrams with expert-level explanations.
  3. Intellectual Property (IP) Privacy: Semiconductor companies cannot risk uploading proprietary designs to cloud-based AI services.

TD-Interpreter addresses these issues by providing a locally deployable solution that captures the "tacit knowledge" of hardware engineering.

Methodology and the Synthetic Pipeline

The development of TD-Interpreter relied on a human-centered design approach, starting with interviews of 31 embedded systems experts. These interviews identified clock-domain crossing (CDC) and multi-clock synchronization as the primary pain points in manual TD analysis.

The researchers utilized LLaVA (Large Language-and-Vision Assistant), a 7B-parameter model, as their base. They applied LoRA (Low-Rank Adaptation) for efficient fine-tuning, enabling the model to learn the visual semantics of waveforms without prohibitive computational costs.

The synthetic data generation pipeline used to train TD-Interpreter.

To overcome the data scarcity problem, the team built an automated synthetic data pipeline. As shown in the figure above, the process begins with Verilog modules and random testbenches. These are simulated to produce Value Change Dump (.vcd) files, which are then converted into images using the industry-standard Wavedrom format. This ensures that every training image is accompanied by a verifiable "ground truth."

Dataset Composition

The resulting dataset is divided into two main categories:

  • Caption-based Data: Approximately 222,000 TDs paired with 10 million simple QA pairs (e.g., identifying signal values at specific cycles).
  • Reasoning-based Data: A curated set of 10,000 high-quality samples focusing on complex logic, including Finite State Machines (FSMs), AHB bus transfers, SPI protocols, and asynchronous FIFOs.

A benchmark of 310 diverse question types was established to evaluate the model's performance on everything from basic timing analysis to complex metastability issues.

Key Results and Performance

The impact of specialized fine-tuning was significant. On the Bleu-4 metric, which measures text similarity, the base LLaVA model scored 13.9, while TD-Interpreter achieved a score of 95.9.

Examples of TD-Interpreter analyzing ADC and FIFO timing diagrams.

As illustrated in the examples above, TD-Interpreter can accurately identify timing events in an ADC interface and reason about data transactions within an asynchronous FIFO. In qualitative comparisons, while GPT-4o often provided generic or incorrect answers, TD-Interpreter correctly analyzed specific timing violations and state transitions. Notably, the model demonstrated the ability to "reverse engineer" FSM descriptions from waveforms, a task that typically requires significant manual effort from engineers.

Conclusion and Future Directions

TD-Interpreter demonstrates that a relatively small, 7B-parameter model can achieve expert-level performance in a niche technical domain when trained on high-quality, synthetic data. By running locally, it satisfies the industry's strict IP privacy requirements.

Future work will focus on expanding the model's robustness to different visual styles (such as raw oscilloscope traces) and integrating Reinforcement Learning from Human Feedback (RLHF) to further refine its reasoning capabilities.

Reference: He, J., et al. "TD-Interpreter: Enhancing the Understanding of Timing Diagrams with Visual-Language Learning." arXiv:2507.16844v1, 2025.

Summary updated

Abstract

We introduce TD-Interpreter, a specialized ML tool that assists engineers in understanding complex timing diagrams (TDs), originating from a third party, during their design and verification process. TD-Interpreter is a visual question-answer environment which allows engineers to input a set of TDs and ask design and verification queries regarding these TDs. We implemented TD-Interpreter with multimodal learning by fine-tuning LLaVA, a lightweight 7B Multimodal Large Language Model (MLLM). To address limited training data availability, we developed a synthetic data generation workflow that aligns visual information with its textual interpretation. Our experimental evaluation demonstrates the usefulness of TD-Interpreter which outperformed untuned GPT-4o by a large margin on the evaluated benchmarks.
BibTeX
@misc{he2025td,
  author = {Jie He and Vincent Theo Willem Kenbeek and Zhantao Yang and Meixun Qu and Ezio Bartocci and Dejan Nickovic and Radu Grosu},
  title = {{TD-Interpreter: Enhancing the Understanding of Timing Diagrams with Visual-Language Learning}},
  year = {2025},
  eprint = {2507.16844},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.16844},
}
Figure