Electrocardiograms (ECGs) are essential for diagnosing arrhythmias and myocardial ischemia, yet the development of multimodal AI systems is limited by a lack of datasets integrating raw signals, diagnostic images, and textual interpretations. We present MEETI (MIMIC-IV-Ext ECG-Text-Image), a large-scale dataset that synchronizes raw waveforms, high-resolution plots, beat-level quantitative parameters, and detailed LLM-generated interpretations for over 780,000 records. MEETI provides a robust foundation for next-generation cardiovascular AI research by enabling cross-modal alignment and explainable reasoning.
Citation: Zhang, D., Lan, X., Geng, S. et al. MEETI: A Multimodal ECG Dataset from MIMIC-IV-ECG with Signals, Images, Features and Interpretations. Sci Data (2026). https://doi.org/10.1038/s41597-026-06796-1
Introduction
Cardiovascular diseases (CVDs) are the leading cause of mortality worldwide. While machine learning has achieved expert-level performance in ECG analysis, most models operate as "black boxes" on 1D time-series data. Modern Multimodal Large Language Models (MLLMs) excel at reasoning over images and text but struggle to directly process temporal waveforms. Bridging this gap requires datasets that jointly represent ECG signals alongside visual and semantic information.
Existing public datasets typically offer only single-modality signals or brief signal-text pairs. MEETI addresses this bottleneck by providing a unified triadic framework—linking waveforms, image features, and domain-specific knowledge—to enable explainable, inference-based reasoning in cardiac AI.
Methods and Pipeline
MEETI is built upon the MIMIC-IV-ECG database, which contains approximately 800,000 10-second, 12-lead ECG recordings. The construction pipeline involves three primary stages to generate the multimodal components.
Using the open-source FeatureDB toolkit, we extracted beat-level quantitative descriptors. The process involves adaptive peak detection and discrete wavelet transforms to localize P, QRS, and T waves, ensuring precise measurement of temporal intervals and amplitudes across all 12 leads. These parameters allow models to ground their diagnoses in explicit, measurable physiological changes.
Extracted ECG parameters and their clinical descriptions.
To support vision-based models, raw signals were rendered into standardized graphical formats using the ecg_plot library. Parameters were set to emulate clinical paper recordings (25 mm/s speed, 10 mm/mV amplitude) at 100 dpi, preserving pixel-level detail of low-amplitude features and ensuring precise spatial encoding of waveform morphology.
3. LLM-Based Interpretation Generation
Standard clinical reports are often too brief for training complex reasoning models. We utilized GPT-4o with role-based prompts, feeding it both the original clinical reports and the extracted numerical parameters. This allowed the generation of high-granularity interpretations that explicitly link diagnoses to measurable physiological data, such as noting a QRS duration > 120ms to justify a conduction disorder diagnosis.
Dataset Characteristics
The MEETI dataset aligns each record across four synchronized components:
Raw Signals: 500 Hz 12-lead waveforms from MIMIC-IV-ECG.
Per-Beat Parameters: Structured .mat files containing detailed metrics like PR interval and QRS duration.
Interpretation Text: Detailed, parameter-grounded diagnostic reports generated by GPT-4o.
The dataset includes 784,680 records from 160,597 patients. Statistical validation confirms that the extracted parameters align with physiological norms, such as a median PR interval of ~140 ms and a median QRS duration of ~90 ms.
Conclusion and Future Work
MEETI shifts the focus of ECG AI from simple end-to-end classification to explainable, multi-perspective analysis. While storage limits on platforms like Zenodo necessitate a phased release of the full image corpus, the complete dataset is hosted on Hugging Face. MEETI serves as a foundational benchmark for training MLLMs to "read" and interpret cardiac health with human-like reasoning, moving the field toward more transparent clinical decision support.
When using this resource, please cite: Zhang, D., Lan, X., Geng, S. et al. MEETI: A Multimodal ECG Dataset from MIMIC-IV-ECG with Signals, Images, Features and Interpretations. Sci Data (2026). https://doi.org/10.1038/s41597-026-06796-1 Key Events: Due to Zenodo's 200GB storage limit, the full MEETI dataset cannot be hosted on the platform. The complete dataset is available on Hugging Face. Multimodal data of approximately 10,000 samples https://zenodo.org/records/18523205 Complete MEETI dataset https://huggingface.co/datasets/PKUDigitalHealth/MEETI Abstract Electrocardiograms (ECGs) are essential for diagnosing arrhythmias, myocardial ischemia, and conduction disorders. While machine learning has achieved expert-level performance in ECG interpretation, the development of clinically deployable multimodal aI systems is limited by the lack of public datasets that integrate raw signals, diagnostic images, and interpretation text. Most existing ECG datasets are single-modality or include, at most, signal-text pairs, restricting real-world applicability. To address this gap, we present MEETI (MIMIC-IV-Ext ECG-Text-Image), the first large-scale dataset that synchronizes raw ECG waveforms, high-resolution plotted images, and detailed textual interpretations generated by large language models. MEEtI also includes beat-level quantitative parameters extracted from each lead, enabling fine-grained analysis and improving model interpretability. Built on the MIMIC-IV-ECG database of over 800,000 recordings, MEETI aligns each record across four components using unique identifiers: (1) raw signals, (2) plotted images, (3) per-beat parameters, and (4) interpretation text. This structure enables multimodal transformer learning and supports explainable, integrated analysis. MEEtI provides a robust foundation and benchmark for next-generation cardiovascular artificial intelligence research. Background Cardiovascular diseases (CVDs) are the leading cause of mortality worldwide [1], contributing to over 17 million deaths each year. Electrocardiography (ECG) remains the primary noninvasive modality for assessing cardiac electrophysiology. It provides temporal waveform information that is critical for diagnosing arrhythmias, myocardial ischemia, and conduction abnormalities [2]. However, manually interpreting large volumes of ECG recordings is time-consuming and prone to inter-observer variability. Recently, machine learning and AI approaches have emerged as powerful tools for ECG analysis [3]. They can achieve expert-level performance in waveform delineation, arrhythmia detection, and risk stratification. These advances underscore the urgent need for comprehensive, multimodal ECG datasets. Such datasets are essential to ensure reproducible research and to accelerate clinical translation. Over the past few years, multimodal large language models have demonstrated remarkable performance in interpreting complex medical imaging data [4]. They can seamlessly integrate radiographic, histopathological, and clinical text inputs to support diagnostic and prognostic workflows. These advanced architectures use transformer-based encoders to fuse pixel-level features with semantic context. As a result, tasks such as lesion detection, disease grading, and report generation can be performed at near-human accuracy. Despite these successes, current large language models remain incapable of directly processing and understanding electrocardiographic waveforms. Cardiac signals have unique temporal dynamics and amplitude variations that differ fundamentally from spatial image patterns. Bridging this gap requires specialized datasets and model designs that jointly represent ECG signals alongside complementary textual and visual information. Given that ECG data inherently combine visual patterns, temporal dynamics, and semantic information, their multimodal nature has long been recognized in both clinical and computational analyses. Traditional ECG pipelines rely on handcrafted feature extraction (e.g., QRS duration, ST-segment deviations) to support diagnostic decisions [5]. In contrast, recent deep learning approaches achieve end-to-end performance gains but are often criticized as "black boxes" lacking explicit interpretability and semantic features [6]. These limitations underscore the need to model ECGs through an integrated, multimodal lens. As a result, we need a unified “ECG multimodal” approach. Training multimodal language models to understand ECG data requires genuine “ECG multimodal” examples—paired waveform images, raw signals, and textual annotations. However, existing public ECG datasets only include signal–text pairs and leave out image data. This gap prevents a systematic study of how to combine visual and temporal cardiac representations in transformer-based models. The lack of integrated datasets is therefore a bottleneck for ECG multimodal research. It motivates the creation of resources that encode images, signals, and semantics together for end-to-end model training. To address these limitations, we introduce MEETI, a novel dataset designed to improve large language models’ understanding of electrocardiograms. MEETI is built on the MIMIC-IV-ECG dataset [7], one of the largest and most representative public 12-lead clinical ECG collections available. It covers various cardiac conditions and signal qualities that reflect real-world clinical settings. MEETI is the first ECG multimodal dataset to include synchronized ECG images, structured diagnostic text, and expert-annotated signal reports. By linking ECG waveforms, image features, and domain-specific knowledge representations, MEETI provides a unified triadic framework. This framework offers a standardized input format for training multimodal large language models and promotes explainable AI by enabling clinically interpretable reasoning rather than “black-box” classification. Integrating visual, temporal, and semantic modalities allows for cross-modal alignment and supports the development of models capable of multi-perspective ECG analysis. In addition, MEETI encourages a shift from traditional end-to-end diagnostic pipelines toward inference-based reasoning. Ultimately, this resource enhances the ability of multimodal large language models to generate meaningful, context-aware interpretations of cardiac health. Methods ECG signals and reports As part of routine clinical care at the Beth Israel Deaconess Medical Center (BIDMC), diagnostic 12-lead ECGs were collected using standard ECG machines and subsequently archived in the MIMIC-IV-ECG [7]. This dataset comprises approximately 800,000 10-second, 12-lead ECG recordings sampled at 500 Hz, matched to nearly 160,000 unique patients included in the broader MIMIC-IV Clinical Database. In addition to the raw waveform data, machine-generated measurements and cardiologist-interpreted reports are also available for a substantial proportion of ECGs. Most ECG waveforms get read by a cardiologist, and an associated report is generated from the reading. All records have been de-identified by the HIPAA Safe Harbor requirements to ensure patient privacy. ECG feature extraction To extract a comprehensive set of quantitative descriptors from the MEETI ECG waveforms, we integrated the open‑source FeatureDB toolkit [5] (https://github.com/PKUDigitalHealth/FeatureDB) into our preprocessing pipeline. FeatureDB employs a multi‐stage workflow beginning with adaptive peak detection to localize the P wave, QRS complex, and T wave across each lead. Following initial peak localization, discrete wavelet transforms are applied at multiple scales to refine the onset and offset boundaries of these waveform components, thereby ensuring precise measurement of key temporal intervals (PR interval, QRS duration, QT interval, and QTc). We applied this pipeline to all de‐identified 12‑lead ECG records after verifying signal integrity (complete lead set, consistent sampling rate, and absence of major artifacts). Extracted features were stored in a structured relational database, with each record linked to its raw WFDB waveform files and corresponding expert reports via unique study identifiers. This systematic application of FeatureDB not only facilitates scalable, reproducible extraction of clinically meaningful ECG metrics but also lays the groundwork for downstream multimodal integration with image and text modalities, ultimately supporting the development of interpretable AI models in cardiac electrophysiology. ECG plot To generate high-fidelity ECG images, we rendered each preprocessed 12‑lead waveform into a standardized graphical format using the open‑source ecg_plot Python library (https://github.com/PKUDigitalHealth/ecg_plot). For each record, we configured the plotting parameters to emulate clinical paper recordings: a paper speed of 25 mm/s and an amplitude scaling of 10 mm/mV. Under these settings, each large grid block corresponds to 0.5 mV in voltage and 0.2 s in time, thereby ensuring precise spatial encoding of waveform morphology. Leads were arranged in the conventional 3×4, 6×2, or 12×1 (I, II, III; aVR, aVL, aVF; V1–V6) with appropriate inter‑panel spacing and grid visibility. Images were exported at 100 dpi resolution to facilitate downstream vision‑based model training and to maintain pixel-level detail of low-amplitude features. Each image file was named using the study identifier, then stored alongside the corresponding waveform and text report for seamless multimodal alignment. This procedural standardization yields a uniform image corpus suitable for transformer-based and convolutional model development in ECG interpretation. ECG interpretation generation Human-written ECG reports are typically brief and often lack the detail needed for comprehensive interpretation. To produce high-granularity ECG interpretations that explicitly link diagnoses to measurable parameters such
BibTeX
@misc{zhang2025meeti,
author = {Deyun Zhang and Xiang Lan and Shijia Geng and Qinghao Zhao and Sumei Fan and Mengling Feng and Shenda Hong},
title = {{MEETI: A Multimodal ECG Dataset from MIMIC-IV-ECG with Signals, Images, Features and Interpretations}},
year = {2025},
doi = {10.5281/zenodo.15893350},
howpublished = {arXiv},
}