Abstract
Multi-lead electrocardiogram (ECG) provides clinical information of heartbeats from several fixed viewpoints determined by the lead positioning. However, it is often not satisfactory to visualize ECG signals in these fixed and limited views, as some clinically useful information is represented only from a few specific ECG viewpoints. For the first time, we propose a new concept, Electrocardio Panorama, which allows visualizing ECG signals from any queried viewpoints. To build Electrocardio Panorama, we present a Neural electrocardio field Network (Nef-Net), which predicts the electrocardio field representation using a sparse set of input ECG views and synthesizes a panorama based on these representations. Experiments verify that Nef-Net performs well on panorama synthesis and outperforms previous work on auxiliary tasks like view transformation and synthesis from scratch.
Introduction: Beyond Fixed Lead Positioning
Standard electrocardiograms (ECGs) are the cornerstone of cardiac diagnostics, yet they are inherently limited by physical electrode placement. Whether using a 3-lead or a 12-lead system, clinicians are restricted to "viewpoints" determined by the standard lead positions on a patient's body. However, the heart's electrical activity is a complex, three-dimensional field. Vital clinical information, such as the exact origin of an arrhythmia, might be most apparent from an angle not captured by standard leads.
To address this, we introduce the Electrocardio Panorama, a framework that treats the heart's electrical activity as a continuous 3D field. This allows for the synthesis of ECG signals from any arbitrary viewpoint $(\theta, \phi)$, effectively allowing clinicians to "rotate" their perspective around the heart.
Methodology: The Nef-Net Architecture
The core of this framework is the Neural electrocardio field Network (Nef-Net). Nef-Net models the heart as an underlying "electrocardio field" that represents the magnitude and direction of electrical signals in 3D space.
Technical Innovations
- Angular Encoding: Drawing inspiration from Neural Radiance Fields (NeRF), we found that raw angular inputs are insufficient for learning complex spatial relationships. We propose a triplet-based encoding $[ \rho, \sin(\rho), \cos(\rho) ]$ for angles, enabling the network to better capture the spatial trigonometric relationships inherent in electrical projections.
- Standin Learning: Since the internal electrocardio field cannot be directly observed (no "ground truth"), we developed a self-supervised approach called Standin Learning. This method uses averaged representations from multiple available views as a target to regularize the model and minimize viewpoint bias.
- 1D ROIAlign for ECGs: To account for variations in heartbeat rhythm and duration, Nef-Net employs a 1D version of ROIAlign. This technique normalizes specific ECG components—such as the P-wave, QRS complex, and T-wave—into a fixed representation space, ensuring consistent processing across different cardiac cycles.
Datasets and Community Contribution
The model was evaluated using three major datasets: MIT-BIH, PTB, and the large-scale Tianchi ECG Dataset (containing over 31,000 12-lead records). A significant contribution of this work is the release of granular cardiac cycle and deflection division labels for the PTB and Tianchi datasets, providing the research community with high-quality metadata for identifying wave boundaries. The codes and labels are available at https://github.com/WhatAShot/Electrocardio-Panorama.
Key Results and Insights
Nef-Net demonstrates a significant leap in signal fidelity and utility:
- Superior View Transformation: In converting sparse leads to full 12-lead sets, Nef-Net outperformed state-of-the-art models like Dower and E-LSTM, improving PSNR by 6–10 points and SSIM by 0.05–0.2.
- High-Fidelity Synthesis: The model produces smooth, realistic transitions as the queried viewpoint rotates, providing a true "panorama" of cardiac activity.
- Enhanced Data Augmentation: When used to generate synthetic data for disease classifiers (e.g., SVEB, VEB), Nef-Net's samples led to higher ROC-AUC scores compared to standard GANs, indicating that the synthesized signals preserve more clinically relevant features.
Conclusion and Future Directions
While Nef-Net represents a breakthrough in ECG visualization, challenges remain. Current amplitude normalization may discard absolute magnitude data relevant to conditions like ventricular hypertrophy. Future work will focus on explicitly incorporating morbid-specific operations and optimizing the system for real-time clinical integration. By disentangling the heart's internal state from the observer's position, Electrocardio Panorama opens new possibilities for personalized cardiac diagnostics.
Summary updated