Backpropagation-free Deep Learning for Time-Series Classification

Miha Kapš, Blaž Bertalanič, Carolina FortunaarXiv 2026 · arXiv

TL;DR

This paper presents the first systematic evaluation of backpropagation-free learning rules—Direct Feedback Alignment (DFA), Direct Random Target Projection (DRTP), and NoProp—for 1D time-series classification. While these methods offer potential escapes from the computational bottlenecks of standard backpropagation, the study reveals a significant "feature learning gap": most current BP-free methods act more like random feature extractors than adaptive learners when applied to convolutional architectures.

What Problem Does This Paper Address?

Deep learning for time-series classification (TSC) is currently dominated by error backpropagation (BP). However, BP comes with two major "physical" constraints:

  1. Update Locking: Layers must be updated sequentially, preventing parallelization and requiring high memory to store intermediate activations.
  2. Weight Transport Problem: BP requires symmetric forward and backward weight matrices, which is biologically implausible and difficult to implement in specialized hardware.

While non-neural methods like ROCKET are incredibly fast, they rely on fixed, random features. The authors wanted to see if BP-free neural methods could bridge this gap—offering the efficiency of random projections with the adaptive power of deep learning—specifically for 1D temporal data, a domain previously unexplored for these algorithms.

Methods

The authors adapted three BP-free paradigms to 1D convolutional architectures:

  • Direct Feedback Alignment (DFA): Bypasses the layer-by-layer backward pass by projecting the global output error directly to each hidden layer through fixed random matrices.
  • Direct Random Target Projection (DRTP): A fully feedforward approach where each layer updates its weights based on a random projection of the target label itself, removing the need to wait for a forward pass to complete.
  • NoProp: Inspired by diffusion models, this method treats learning as a denoising problem. Each block independently learns to denoise a noisy version of the target label, conditioned on the input data.

To test if these models actually "learn" features, the authors used a readout intervention. They compared a position-sensitive "Flatten" readout (which preserves all time-step info) against "Global Average Pooling" (GAP), which requires the network to have learned meaningful, position-invariant features to succeed.

Fig. 1: Comparison of different error propagation strategies. Black arrows indicate feedforward pathways and red arrows indicate feedback pathways. Input time series is denoted as X, target vector as y∗, and loss function as L. (a) Backpropagation: Firstly, the forward pass is performed and error gradients are calculated sequentially for each layer. (b) Direct Feedback Alignment: after forward pass, error is projected independently to each layer. (c) Direct Random Target Projection: target vector is projected to each layer during the forward pass. (d) NoProp: backpropagation occurs in each block independently. There is no gradient flow between the blocks.

Datasets

The study utilized 112 univariate datasets from the UCR time series archive, a standard benchmark in the field. The datasets vary widely in length (15 to 2,844 steps), number of classes (2 to 60), and training set size.

Key Insights and Ideas

  • The Feature Learning Bottleneck: A major "aha moment" was discovering that DFA and DRTP fail to learn task-specific convolutional features. When the "Flatten" readout was replaced with GAP, their performance plummeted compared to a BP control. This suggests they behave like "random-feature machines" (similar to ROCKET) rather than true deep learners.
  • NoProp is the Exception: Unlike the others, NoProp tolerated the pooling intervention just as well as backpropagation, indicating it successfully performs genuine feature learning at the block level.
  • Intra-class Variability is the Enemy: The authors identified that "intra-class temporal variability" (how much samples of the same class differ from each other) is the strongest predictor of why these methods fall short of state-of-the-art models like InceptionTime.
  • Conditional Benefits: The "speed" of BP-free learning is a myth in some contexts. For example, DRTP only becomes faster than BP on sequences longer than 700 time steps.
Fig. 2: Architecture of NoProp network. The input z0 is sampled from a Gaussian noise distribution and is iteratively transformed into z1, . . . , zT via a sequence of learned functions u1, . . . , uT . Each transformation block incorporates the input time series X as a conditioning signal, guiding the representations through successive refinements until a final class prediction ˆy is produced.

Key Results

  • Accuracy: Optimized BP-free learners achieved mean F1 scores between 0.76 and 0.79. This is respectable but remains below ROCKET (0.83) and InceptionTime (0.84).
  • Efficiency (DRTP): DRTP reduced training floating-point operations (FLOPs) by 33% compared to standard BP on the same architecture.
  • Memory (NoProp): NoProp reduced peak memory consumption by 38%, though it was 48% slower to train due to its sequential block-wise implementation in this study.
  • DFA Limitations: In the evaluated configurations, DFA provided almost no computational advantage over backpropagation because its optimized version ended up freezing the convolutional layers entirely.

Limitations and Future Work

The primary limitation is the performance gap; none of the BP-free methods matched the accuracy or speed of the ROCKET baseline. The study also highlights a fundamental failure of DFA and DRTP to align convolutional weights with random feedback, a phenomenon known as the "bottleneck effect."

Future work could focus on:

  • Developing new feedback mechanisms that better handle the weight-sharing constraints of convolutions.
  • Optimizing NoProp for parallel block training to realize its theoretical speed advantages.
  • Exploring these methods on multivariate time series or larger-scale datasets.

Summary updated

Abstract

Deep neural networks for time series classification (TSC) are trained almost exclusively by error backpropagation, which imposes update locking and the weight-transport problem. Backpropagation-free learning rules such as Direct Feedback Alignment (DFA), Direct Random Target Projection (DRTP), and the diffusion-inspired NoProp relax these constraints but have been evaluated almost exclusively on 2D image benchmarks. We adapt these three learners to univariate TSC with 1D convolutional architectures and present the first systematic study of them across 112 datasets from the UCR time series archive, with one hyperparameter-optimized configuration per method. The optimized learners reach mean F1 scores of 0.76 to 0.79, below ROCKET (0.83) and InceptionTime (0.84). We propose an analysis framework that relates the per-dataset gap to InceptionTime to dataset properties and tests convolutional feature learning by replacing the flatten readout with global average pooling, measured as a difference-in-differences (DD) against a backpropagation control on the identical architecture. Intra-class temporal variability is the dataset property most consistently associated with the gap for all three learners (partial Spearman $ρ$ from $0.28$ to $0.40$). Pooling harms DFA and DRTP more than backpropagation (DD $-0.025$ and $-0.055$), consistent with near-random convolutional features, whereas NoProp tolerates pooling as backpropagation does (DD $+0.013$). Relative to backpropagation on the same architectures, DRTP reduces training floating-point operations (FLOPs) by 33% and trains faster only on series longer than about 700 time steps, DFA yields no computational reduction, and NoProp reduces peak memory by 38% at the cost of 48% longer training.
BibTeX
@misc{kaps2026backpropagation,
  author = {Miha Kapš and Blaž Bertalanič and Carolina Fortuna},
  title = {{Backpropagation-free Deep Learning for Time-Series Classification}},
  year = {2026},
  eprint = {2610.41805},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.41805},
}
Figure