Abstract
Deep learning models for inverse problems often struggle with generalization due to "brittleness" when encountering out-of-distribution data or irrelevant variations. This paper proposes a sequence-to-sequence architecture that integrates a stochastic latent space with the Information Bottleneck (IB) principle. By forcing the model to compress input information and represent it stochastically, the network learns to ignore nuisance variables like anatomical variations. Evaluated on cardiac transmembrane potential (TMP) reconstruction from ECG signals, the method demonstrates superior robustness to unseen pathologies and geometric rotations compared to deterministic models and traditional algorithms.
Introduction
Deep learning has revolutionized image reconstruction, yet these models often suffer from a significant "brittleness" problem: they perform exceptionally well on data similar to their training set but fail when faced with novel scenarios. In medical imaging, this is a major hurdle. For instance, a model trained to reconstruct the heart's electrical activity—transmembrane potential (TMP)—from body-surface ECGs might fail if a patient has a slightly different torso shape or a heart pathology not represented in the training data.
The authors address two specific types of generalization failures:
- Out-of-distribution data: Discrepancies in the underlying signal, such as specific types of heart scars.
- Irrelevant variations: Nuisance information in the input, such as the specific rotation of the heart inside the chest, which is not required for reconstructing the target signal.
Methodology
The proposed solution involves an encoder-decoder architecture designed for sequences, utilizing Long Short-Term Memory (LSTM) networks. The core innovation lies in the integration of two theoretical concepts: a stochastic latent space and the Information Bottleneck principle.
Stochastic Latent Space
Instead of representing the "essence" of a sequence as a fixed vector, the encoder predicts the parameters (mean and variance) of a probability distribution. During training, the model samples from this distribution. Drawing from analytical learning theory, the authors demonstrate that this stochasticity forces the decoder to be "flatter" and less sensitive to small perturbations, naturally enhancing generalization.
Information Bottleneck (IB) Principle
The model is trained with a specialized loss function that balances reconstruction accuracy against the amount of information compressed into the latent space. By "squeezing" information through this bottleneck, the network is forced to discard features irrelevant to the final output—such as heart-torso geometry—while retaining essential signal characteristics.
Architecture Variants
The study compares a standard sequence-to-sequence model (svs-L), which utilizes only the last hidden state of the LSTM, against a proposed svs architecture. The svs model aggregates all hidden states from the sequence into the latent representation, providing a more comprehensive global temporal context.
Experimental Setup
Datasets
- Synthetic Data: The researchers utilized the Aliev-Panfilov (AP) model to simulate 2,500 pairs of TMP and ECG data across three distinct human-torso geometries. Test sets were categorized into four levels of difficulty, varying the origin of heart excitation and the presence of myocardial scars.
- Real-World Data: To demonstrate clinical feasibility, the model was tested on clinical 120-lead ECG data from a patient with scar-related ventricular tachycardia. Results were validated against in-vivo voltage maps obtained during physical procedures.
Results
Superior Generalization
On the most challenging "unseen pathology" test cases, the stochastic svs model achieved an Activation Time (AT) Correlation of 0.885. In stark contrast, its deterministic counterpart collapsed, achieving a correlation of only 0.12.
Geometric Robustness
The stochastic IB model maintained high accuracy even when the heart was rotated up to 15 degrees—a variation irrelevant to the TMP signal. Standard models exhibited a sharp increase in error under the same conditions, proving the effectiveness of the information bottleneck in filtering nuisance variables.
Clinical Validation
Despite being trained entirely on simulated data, the model successfully identified scar regions in a real patient that closely matched clinical voltage maps. It also significantly outperformed the "Greensite" method, a traditional gold-standard algorithm for cardiac inverse problems.
Conclusion
This work demonstrates that incorporating stochasticity and information-theoretic constraints into deep learning architectures can significantly improve their robustness for medical inverse problems. By focusing on essential features and ignoring anatomical nuisances, these models move closer to reliable clinical application. Future work will explore learning the prior distribution for the Information Bottleneck and applying these principles to other modalities like MRI and CT.
Summary updated