Electrocardio Panorama: Synthesizing New ECG Views with Self-supervision

Jintai Chen, Xiangshang Zheng, Hongyun Yu, Danny Z. Chen, Jian Wu

Abstract

Multi-lead electrocardiogram (ECG) provides clinical information of heartbeats from several fixed viewpoints determined by the lead positioning. However, it is often not satisfactory to visualize ECG signals in these fixed and limited views, as some clinically useful information is represented only from a few specific ECG viewpoints. For the first time, we propose a new concept, Electrocardio Panorama, which allows visualizing ECG signals from any queried viewpoints. To build Electrocardio Panorama, we present a Neural electrocardio field Network (Nef-Net), which predicts the electrocardio field representation using a sparse set of input ECG views and synthesizes a panorama based on these representations. Experiments verify that Nef-Net performs well on panorama synthesis and outperforms previous work on auxiliary tasks like view transformation and synthesis from scratch.

Introduction: Beyond Fixed Lead Positioning

Standard electrocardiograms (ECGs) are the cornerstone of cardiac diagnostics, yet they are inherently limited by physical electrode placement. Whether using a 3-lead or a 12-lead system, clinicians are restricted to "viewpoints" determined by the standard lead positions on a patient's body. However, the heart's electrical activity is a complex, three-dimensional field. Vital clinical information, such as the exact origin of an arrhythmia, might be most apparent from an angle not captured by standard leads.

The concept of Electrocardio Panorama: (a) Multi-lead ECG, (b) Lead positioning, (c) An ECG view, and (d) Electrocardio Panorama with a queried viewpoint.

To address this, we introduce the Electrocardio Panorama, a framework that treats the heart's electrical activity as a continuous 3D field. This allows for the synthesis of ECG signals from any arbitrary viewpoint $(\theta, \phi)$, effectively allowing clinicians to "rotate" their perspective around the heart.

Methodology: The Nef-Net Architecture

The core of this framework is the Neural electrocardio field Network (Nef-Net). Nef-Net models the heart as an underlying "electrocardio field" that represents the magnitude and direction of electrical signals in 3D space.

The Nef-Net architecture for synthesizing ECG views from queried angles using sparse input views and self-supervised Standin Learning.

Technical Innovations

  • Angular Encoding: Drawing inspiration from Neural Radiance Fields (NeRF), we found that raw angular inputs are insufficient for learning complex spatial relationships. We propose a triplet-based encoding $[ \rho, \sin(\rho), \cos(\rho) ]$ for angles, enabling the network to better capture the spatial trigonometric relationships inherent in electrical projections.
  • Standin Learning: Since the internal electrocardio field cannot be directly observed (no "ground truth"), we developed a self-supervised approach called Standin Learning. This method uses averaged representations from multiple available views as a target to regularize the model and minimize viewpoint bias.
  • 1D ROIAlign for ECGs: To account for variations in heartbeat rhythm and duration, Nef-Net employs a 1D version of ROIAlign. This technique normalizes specific ECG components—such as the P-wave, QRS complex, and T-wave—into a fixed representation space, ensuring consistent processing across different cardiac cycles.

Datasets and Community Contribution

The model was evaluated using three major datasets: MIT-BIH, PTB, and the large-scale Tianchi ECG Dataset (containing over 31,000 12-lead records). A significant contribution of this work is the release of granular cardiac cycle and deflection division labels for the PTB and Tianchi datasets, providing the research community with high-quality metadata for identifying wave boundaries. The codes and labels are available at https://github.com/WhatAShot/Electrocardio-Panorama.

Key Results and Insights

Nef-Net demonstrates a significant leap in signal fidelity and utility:

  • Superior View Transformation: In converting sparse leads to full 12-lead sets, Nef-Net outperformed state-of-the-art models like Dower and E-LSTM, improving PSNR by 6–10 points and SSIM by 0.05–0.2.
  • High-Fidelity Synthesis: The model produces smooth, realistic transitions as the queried viewpoint rotates, providing a true "panorama" of cardiac activity.
  • Enhanced Data Augmentation: When used to generate synthetic data for disease classifiers (e.g., SVEB, VEB), Nef-Net's samples led to higher ROC-AUC scores compared to standard GANs, indicating that the synthesized signals preserve more clinically relevant features.

Conclusion and Future Directions

While Nef-Net represents a breakthrough in ECG visualization, challenges remain. Current amplitude normalization may discard absolute magnitude data relevant to conditions like ventricular hypertrophy. Future work will focus on explicitly incorporating morbid-specific operations and optimizing the system for real-time clinical integration. By disentangling the heart's internal state from the observer's position, Electrocardio Panorama opens new possibilities for personalized cardiac diagnostics.

Summary updated

Abstract

Multi-lead electrocardiogram (ECG) provides clinical information of heartbeats from several fixed viewpoints determined by the lead positioning. However, it is often not satisfactory to visualize ECG signals in these fixed and limited views, as some clinically useful information is represented only from a few specific ECG viewpoints. For the first time, we propose a new concept, Electrocardio Panorama, which allows visualizing ECG signals from any queried viewpoints. To build Electrocardio Panorama, we assume that an underlying electrocardio field exists, representing locations, magnitudes, and directions of ECG signals. We present a Neural electrocardio field Network (Nef-Net), which first predicts the electrocardio field representation by using a sparse set of one or few input ECG views and then synthesizes Electrocardio Panorama based on the predicted representations. Specially, to better disentangle electrocardio field information from viewpoint biases, a new Angular Encoding is proposed to process viewpoint angles. Also, we propose a self-supervised learning approach called Standin Learning, which helps model the electrocardio field without direct supervision. Further, with very few modifications, Nef-Net can also synthesize ECG signals from scratch. Experiments verify that our Nef-Net performs well on Electrocardio Panorama synthesis, and outperforms the previous work on the auxiliary tasks (ECG view transformation and ECG synthesis from scratch). The codes and the division labels of cardiac cycles and ECG deflections on Tianchi ECG and PTB datasets are available at https://github.com/WhatAShot/Electrocardio-Panorama.
BibTeX
@inproceedings{chen2021electrocardio,
  author = {Jintai Chen and Xiangshang Zheng and Hongyun Yu and Danny Z. Chen and Jian Wu},
  title = {{Electrocardio Panorama: Synthesizing New ECG Views with Self-supervision}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/495},
}
Figure