Aligning Temporal Delays in Branching Networks

17 concepts8 papers1 proposal

In standard artificial neural networks (ANNs), we treat information propagation as instantaneous. When evaluating a residual block, the operation $y = x + f(x)$ assumes that the identity feature $x$ and the transformed feature $f(x)$ exist in the same timeless snapshot.

However, when neural networks are deployed in frameworks where time is an explicit dimension—such as Spiking Neural Networks (SNNs), real-time streaming video models, or dynamical learning systems like Predictive Coding—this assumption breaks down. Computing $f(x)$ takes physical or simulation time, whereas the shortcut path $x$ delivers information instantly. This structural asymmetry creates a branch timing discrepancy: signals traveling through shorter paths arrive too early, while signals traveling through deeper paths arrive too late.

Without explicit synchronization, this asynchronous feature arrival leads to premature neuron firing, destructive interference of error signals, and severe training instability. A growing body of research focuses on mechanisms to track, buffer, and synchronize these parallel paths to restore architectural coherence.


The Intuition: The River Fork Paradox

To understand why asynchronous arrival degrades network performance, imagine a river that forks into two channels: a straight, high-speed canal (the shortcut) and a winding, slow stream with water wheels (the deep convolutional path). If you drop two pieces of a puzzle into both channels simultaneously, the piece in the canal will reach the confluence long before the piece in the winding stream. If the downstream station merges whatever arrives at any given moment, it will mistakenly combine the canal's new puzzle piece with the winding stream's old puzzle piece.

This exact misalignment happens across three major paradigms when branching architectures like ResNets or Inception blocks are introduced to temporal domains.

Shortcut Path:      [Input x] -----------------------------> (Early Arrival) ---\
                                                                                [Merge Node]
Convolutional Path: [Input x] --> [Conv] --> [Conv] ------> (Late Arrival) ----/

1. Spiking Neural Networks: Preventing Premature Activation

In Spiking Neural Networks utilizing Time-To-First-Spike (TTFS) coding, information is encoded in the exact arrival time of a spike; earlier spikes signify higher importance or intensity.

When a skip connection is introduced, the shortcut path delivers spikes almost immediately. As Youngeun Kim et al. demonstrate in Rethinking skip connections in Spiking Neural Networks with Time-To-First-Spike coding, this rapid propagation causes downstream neurons to fire prematurely. The earliest-arriving signal triggers the neuron before the richer, more complex features from the slower convolutional path even arrive, effectively blinding the network to deep abstractions. To solve this, the authors introduce a learnable delay parameter directly into the skip connection, forcing the shortcut signals to wait until the convolutional features catch up.

A similar synchronization failure occurs when converting pre-trained continuous ANNs into spiking equivalents. In BSNN: Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons, Yang Li et al. explain that signals propagate at vastly different speeds across parallel paths during spiking simulation. By implementing temporal buffering, the authors align signals from paths of different lengths, ensuring they reach the destination layer simultaneously and preserving the structural integrity of the original architecture.

Further validating this approach, Siddiqui Muhammad Yasir et al. demonstrate in BN-SNN: Spiking neural networks with bistable neurons for object detection that integrating synchronous neurons within residual blocks mitigates these phase discrepancies, stabilizing the network during complex tasks like object detection.


2. Predictive Coding: Eliminating Destructive Interference

Predictive Coding (PC) is a biologically plausible alternative to backpropagation where layers update their weights locally by minimizing prediction errors over time. Because PC treats the network as a continuous dynamical system, error signals propagate as physical waves through the architecture.

When skip connections are present, error signals traveling backward encounter paths of varying lengths. In Reverse Differentiation via Predictive Coding, Tommaso Salvatori et al. show that this causes error signals to reach specific nodes at staggered times. This temporal misalignment prevents local learning rules from accurately calculating global gradients. The authors resolve this by inserting identity nodes into the network to equalize path lengths, ensuring perfect synchronization of the error waves.

Without this structural alignment, deep predictive networks suffer from severe training instability. As Chang Qi et al. articulate in Towards the Training of Deeper Predictive Coding Neural Networks, the latency mismatch between the shortcut and the main path causes destructive interference between the converging error signals. By integrating buffer neurons into the skip connections, the authors delay the faster signals, neutralizing the phase mismatch and allowing predictive coding to scale to deep architectures like ResNet-18.

Asynchronous Error Waves (Destructive Interference):
Shortcut Error:    ---\___/---
Main Path Error:   ___/---\___  --> Cancel each other out!

Synchronized Error Waves (Constructive Alignment via Buffers):
Buffered Shortcut: ___/---\___
Main Path Error:   ___/---\___  --> Correct Gradient Update

3. Continual Streaming Inference: Maintaining Temporal Coherence

In real-time online inference—such as frame-by-frame video recognition—temporal operations like 3D convolutions or temporal pooling introduce explicit causal delays. For example, a 3D convolution with a temporal kernel size of 3 must look at frames $t-2$, $t-1$, and $t$ to compute its output at time $t$. This introduces a processing latency.

If this convolutional layer is paired with a standard residual shortcut, the shortcut passes frame $t$ instantly, while the convolutional path outputs a feature map representing an accumulation up to frame $t$. Merging them directly means adding a feature map from the past to a frame from the present, destroying temporal coherence.

In Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch, Lukas Hedegaard et al. introduce an automated framework that tracks the exact latency introduced by causal operations across branching structures. The library automatically computes and injects compensating delays into the shortcut paths.

As the authors demonstrate in a companion work on Continual 3D Convolutional Neural Networks for Real-time Processing of Videos, designing residual connections and pooling layers with these precise temporal delays ensures that all converging feature maps represent the exact same point in time, enabling highly efficient, frame-by-frame streaming without accuracy loss.


Embracing Asynchrony: An Alternative Path

While the dominant solution is to force alignment via buffers and delays, an alternative perspective is to make the network's learning rules resilient to asynchrony.

In Learning From the Past with Cascading Eligibility Traces, Tokiniaina Raharison Ralambomihanta et al. tackle varying feedback delays by assigning different temporal look-back windows to different network layers. Instead of forcing signals to arrive at the same instant, this approach allows earlier layers to remain stable by holding a temporally precise "memory bump" of past states, allowing them to successfully integrate feedback signals that arrive significantly later than those received by output layers.

Summary

Whether preventing premature spikes in neuromorphic hardware, stopping destructive interference in local learning rules, or aligning video frames in real-time streaming, temporal path synchronization solves a fundamental flaw of branching architectures in dynamical environments. By treating time as a first-class citizen and explicitly compensating for path length discrepancies, these techniques bridge the gap between static network design and the temporal reality of physical computation.

Key Foundational Work

BSNN: Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons (2022) — contributed 3 concept elements to this direction.
Rethinking skip connections in Spiking Neural Networks with Time-To-First-Spike coding (2024) — contributed 2 concept elements to this direction.
Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch (2022) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_path_synchronization_73b23d79,
  title = {temporal path synchronization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-path-synchronization/rdr_73b23d792f5b4cf3ad41227355927671},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_temporal_path_synchronization_73b23d79_v1,
  title = {temporal path synchronization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-path-synchronization/rdr_73b23d792f5b4cf3ad41227355927671/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake