Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning

Aristotelis Ballas, Vasileios Papapanagiotou, Christos Diou

TL;DR

Self-supervised contrastive learning can make heart-sound (phonocardiogram) classification models highly robust to new clinical environments, but its success hinges entirely on choosing the right data augmentations. By evaluating roughly 150 augmentation combinations across 4,000 models, this paper provides a definitive recipe for robust biosignal processing, showing that low-pass filtering and random flipping are crucial for learning generalizable audio representations.

What Problem Does This Paper Address?

Automating the detection of heart abnormalities using phonocardiograms (PCGs)—recordings of heart sounds—could revolutionize early cardiovascular care. However, deep learning models in this domain face a major hurdle: distribution shift. A model trained on data from one specific clinic often fails when deployed in another due to differences in microphones, ambient noise, patient body mass, or sensor placement.

While Self-Supervised Learning (SSL) via contrastive frameworks (like SimCLR) has proven excellent at learning robust, out-of-distribution (OOD) representations in computer vision, its application to 1D medical audio signals remains under-explored. Crucially, contrastive learning relies on data augmentations to create "views" of the data, but standard image augmentations (like cropping or color jittering) do not translate to audio waves. This paper systematically investigates which audio augmentations actually help or hinder a model's ability to generalize to unseen clinical environments.

Methods

The authors adapted the popular SimCLR contrastive learning framework for 1D PCG signals. The pipeline consists of a six-step process to pretrain a 5-layer 1D Convolutional Neural Network (CNN) backbone on unlabeled data before fine-tuning it on labeled datasets.

FIGURE 1: Illustration of the proposed experiment pipeline for training and evaluating the effectiveness and robustness of a model trained via Self-Supervised Contrastive Learning for PCG classification. The framework has six steps. In the first step, all datasets are prepared and homogenized into a common format, as described in Section III-A. In the second step, each signal is augmented to two versions. In the next step, the unlabeled and augmented signals are used to train the backbone encoder (Fig. 2 left) to maximize the agreement between representations originating from the same initial signal. Following the pretraining phase, in step 4, a classification head (Fig. 2 right) is attached to the frozen weights of the pretrained encoder. The classifier weights are fine-tuned on data drawn from one of the Pascal, PhysioNet2016 or PhysioNet2022 datasets, in a fully supervised manner. The final model is then evaluated on the test split of the same dataset. In the fifth step of the framework, the generalization ability of the fine-tuned model is assessed on signals drawn from datasets which are left-out ("unseen") during the training process in step 4. We argue that given sufficient data and appropriate augmentations, the pretrained encoder will be able to extract a generalized PCG representation regardless the dataset. Finally, we repeat the fine-tuning and OOD evaluation steps for each remaining dataset (step 6), to assess the robustness of our method when the classifier has been trained on different signal distributions.

The core contribution is an exhaustive empirical evaluation of 6 distinct signal transformations, mixed and matched in various composition schemes (0vs1, 1vs1, 1vs2, 2vs2):

  1. Gaussian Noise: Adding zero-mean noise.
  2. Cut-off Filters: High-pass or low-pass filtering at 250, 500, or 750 Hz.
  3. Scale: Randomly scaling the signal amplitude.
  4. Reverse: Reversing the time sequence.
  5. Inversion: Multiplying the signal by -1.
  6. Random Flip: Concurrently applying reverse and inversion with a given probability.

To move beyond simple trial-and-error, the authors calculated the Cohen’s $d$ effect size for each transformation, statistically quantifying exactly how much an augmentation contributes to OOD generalization.

FIGURE 2: The architecture of the CNN encoder trained via SSL contrastive learning (left) and the classification head trained for the downstream task (right). The weights of the encoder or "backbone network" are frozen after the pretrain- ing phase. During the downstream task, the classification head is attached to the pretrained encoder and its weights are fine-tuned on the dataset of said task in a fully-supervised manner. The architecture of the "downstream model" is used as a baseline in all of our experiments, where all of its layer weights are adjusted during fully-supervised training.

Datasets

The study leverages five distinct public PCG databases to rigorously test cross-dataset generalization:

  • Unlabeled Pretraining: Ephnogram (multi-channel adult recordings during rest/exercise) and the Fetal PCG Database (FPCGDB).
  • Labeled Downstream & OOD Evaluation: Pascal (832 signals, multi-class), PhysioNet2016 (3,126 signals, binary normal/abnormal), and PhysioNet2022 (3,163 signals, multi-class murmur detection).

Key Insights and Ideas

  • Low-Pass Filters are Essential: Applying a low-pass filter (especially with a 500Hz or 250Hz cutoff) yielded a massive positive effect size ($d > 1.0$). This aligns with cardiac physiology: critical heart sound components reside in lower frequencies, while ambient environmental noise dominates higher frequencies. Low-pass filtering forces the network to focus on the invariant biological signal.
  • Random Flip is a Secret Weapon: Combining sequence reversal and inversion (Random Flip with $p=0.5$) showed the highest overall positive effect size ($d \approx 1.8$).
  • More Noise is Not Better: Adding artificial Gaussian noise or applying high-pass filters actively degraded performance ($d < 0$). Because clinical PCG data is already inherently noisy, adding more noise renders the signal meaningless, while high-pass filtering strips away the actual heart sounds.
FIGURE 3: Number of occurrences of each augmentation type in the top 75 performing models in each downstream task, leading to 150 total augmentation pairs. This offers and indication on the importance of each augmentation across all experiments. The plot on the left presents the result for In-Distribution experiments, while the plot on the right the results for the OOD experiments.

Key Results

  • Supervised Fragility vs. SSL Resilience: When evaluated on completely unseen, out-of-distribution datasets, fully-supervised models experienced up to a 32% drop in effectiveness. Conversely, SSL-pretrained models lost only around 10%, and in some cases, their performance actually improved on OOD data.
  • OOD Superiority: On average, across all evaluation cycles, the proposed SSL framework outperformed fully-supervised baselines by 11.68% on OOD data, reaching a peak improvement of 21.04% on the PhysioNet2022 OOD task.
  • Data Scarcity Mitigation: On the small and notoriously difficult Pascal dataset, the SSL model significantly outperformed the baseline even on in-distribution data, proving that SSL acts as an effective regularizer when labeled data is scarce.

Limitations and Future Work

While highly effective, the authors note that contrastive SSL comes with a steep computational constraint: it requires a massive corpus of unlabeled data to maximize representation learning and relies heavily on large training batch sizes (e.g., $B=256$, effectively $512$ after augmentation).

Moving forward, the research team plans to explore more complex, domain-specific audio transformations and extend this exact empirical benchmarking protocol to other critical 1D biosignals, such as Electrocardiograms (ECG), Electromyograms (EMG), and Electroencephalograms (EEG).

Summary updated

Abstract

Despite recent advancements in deep learning, its application in real-world medical settings, such as phonocardiogram (PCG) classification, remains limited. A significant barrier is the lack of high-quality annotated datasets, which hampers the development of robust, generalizable models that can perform well on newly collected, out-of-distribution (OOD) data. Self-Supervised Learning (SSL) contrastive learning, has shown promise in mitigating the issue of data scarcity by using unlabeled data to enhance model robustness. Even though SSL methods have been proposed and researched in other domains, works focusing on the impact of data augmentations on model robustness for PCG classification are limited. In particular, while augmentations are a key component in SSL, selecting the most suitable policy during training is highly challenging. Improper augmentations can lead to substantial performance degradation and even hinder a network's ability to learn meaningful representations. Addressing this gap, our research aims to explore and evaluate a wide range of audio-based augmentations and uncover combinations that enhance SSL model performance in PCG classification. We conduct a comprehensive comparative analysis across multiple datasets, assessing the impact of various augmentations on model performance. Our findings reveal that depending on the training distribution, augmentation choice significantly influences model robustness, with fully-supervised models experiencing up to a 32\% drop in effectiveness when evaluated on unseen data, while SSL models demonstrate greater resilience, losing only 10\% or even improving in some cases. This study also highlights the most promising and appropriate augmentations for PCG signal processing, by calculating their effect size on training. These insights equip researchers with valuable guidelines for developing reliable models in PCG signal processing.
BibTeX
@inproceedings{ballas2024which,
  author = {Aristotelis Ballas and Vasileios Papapanagiotou and Christos Diou},
  title = {{Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning}},
  booktitle = {IEEE Access},
  year = {2024},
  url = {https://openreview.net/forum?id=1UUZ1cIrJ2},
}
Figure