Abstract
The development of low-power wearable systems requires specialized techniques to accommodate their unique requirements and constraints. While significant advancements have been made in the inference phase of artificial intelligence, the training phase remains a challenge, particularly for biomedical wearable systems. Traditional training algorithms might not be suitable for these applications due to the substantial memory requirements and high computational costs associated with processing the large number of bits involved in neural network operations. In this paper, we introduce a novel learning procedure specifically designed for low-power wearable systems, dubbed Bio-BPfree (deep neural network training without backpropagation for low-power wearable systems). Using a two-class classification task, Bio-BPfree replaces conventional forward and backward backpropagation passes with four forward passes, two for data of the positive class and two for data of the negative class. Each layer is equipped with a unique objective function aimed at minimizing the distance between data points within the same class while maximizing the distance between data points from different classes. Our experimental results, which were obtained by conducting rigorous evaluations on the MIT-BIH dataset that features electrocardiogram (ECG) signals, effectively demonstrate the superior performance and suitability of Bio-BPfree for two-class classification tasks, particularly within the challenging environment of low-power wearable systems designed for continuous health monitoring and assessment.
Introduction
Training deep neural networks (DNNs) is notoriously resource-intensive. While "inference" (using a pre-trained model) has become efficient on small devices, on-device training remains a massive hurdle for biomedical wearables like heart monitors. Traditional training relies on backpropagation (BP), which requires a "backward pass" where gradients are sent from the end of the network back to the beginning. This creates two major problems for wearables: significant memory bloat, as the device must store all intermediate activations, and high latency, as layers must wait for subsequent ones to finish before updating weights.
Bio-BPfree is a novel training framework that enables DNNs to learn directly on low-power wearable devices by eliminating backpropagation entirely. By using local, distance-based loss functions for each layer, it reduces memory requirements by up to 99% and computational costs by over 80% while maintaining performance comparable to traditional methods.
Methods: Local Learning without Backpropagation
The core innovation of Bio-BPfree is a layer-wise learning strategy that replaces the global backward pass with local objectives.
The Training Procedure
Instead of one global error signal, each layer in the network has its own local objective. For every training iteration, a layer processes four samples: two from the positive class (e.g., a normal heartbeat) and two from the negative class (e.g., an arrhythmia). The layer is trained to minimize the distance between samples of the same class and maximize the distance between samples of different classes in its feature space. Layers are trained sequentially; once a layer is optimized to cluster these features effectively, its weights are frozen, and the process moves to the next layer.
Architecture and Normalization
The framework was implemented on Res1DCNN, a model with 13 convolutional layers designed for 1D signal processing. To ensure that the first layer does not perform all the feature extraction, a normalization step is implemented between layers. This forces subsequent layers to identify new, relative patterns in the data rather than relying on the absolute magnitude of the previous layer's output.
Experimental Setup
The framework was evaluated using the PhysioNet MIT-BIH Arrhythmia database, consisting of ECG signals from 48 subjects. The task was binary classification of heartbeats (Class N vs. Class V). ECG waves were normalized and split into individual heartbeats without aggressive filtering to simulate real-world wearable conditions.
Results and Discussion
Bio-BPfree proves that complex global derivatives are not strictly necessary to learn useful features. Simply instructing a layer to "keep similar things together and push different things away" is sufficient to build a powerful classifier.
Key Performance Metrics
- Memory Savings: Traditional backpropagation required 17.81 MB of memory. Bio-BPfree’s heaviest layer required only 3.01 MB, and its lightest just 0.095 MB—a reduction of up to 99.47%.
- Computational Speedup: Bio-BPfree reduced the computational load (FLOPs) by 82.8% for the heaviest layers.
- Classification Performance: The model achieved a median sensitivity of 97.67%, outperforming some state-of-the-art backpropagation-based models in identifying specific cardiac events.
Conclusion and Future Work
Bio-BPfree offers a biologically plausible approach to on-device learning, closer to how humans process local sensory observations. While currently optimized for binary classification, future work will focus on expanding the framework to multi-class problems and exploring distributed learning, where different wearable devices could each train different layers of a shared model. Despite a slight trade-off in specificity compared to full-scale backpropagation, the massive resource savings make it a premier candidate for continuous health monitoring in power-constrained environments.
Summary updated