Abstract
Traditional training of Graph Neural Networks (GNNs) relies heavily on the Backpropagation (BP) algorithm, which faces challenges regarding biological plausibility and sequential execution bottlenecks. This paper introduces DFA-GNN, a novel training framework that adapts Direct Feedback Alignment (DFA) for the non-Euclidean domain of graphs. By utilizing graph-aware feedback links and a pseudo-error generation mechanism for unlabeled nodes, DFA-GNN enables parallel layer updates and achieves superior accuracy and robustness compared to standard BP and existing non-BP alternatives.
Introduction
Most modern deep learning architectures, including Graph Neural Networks (GNNs), are trained using Backpropagation (BP). While BP is highly effective, it is increasingly scrutinized for two primary reasons. First, it lacks biological plausibility; the human brain does not appear to utilize the "weight symmetry" (using the same weights for forward and backward passes) or "update locking" (sequential dependencies between layers) required by BP. Second, these sequential dependencies create efficiency bottlenecks, limiting the potential for massive parallelization across network layers.
While alternative "forward learning" methods like Direct Feedback Alignment (DFA) have been proposed for standard Euclidean data (like images), they typically fail when applied to graphs. Graphs present unique challenges: data points (nodes) are interconnected rather than independent, and in semi-supervised settings, ground-truth labels are available for only a small fraction of the data. DFA-GNN addresses these hurdles by redesigning the feedback mechanism to respect graph topology and handle sparse supervision.
Methodology
DFA-GNN replaces the traditional backward pass with a direct projection of global errors to every hidden layer. The framework consists of four core innovations:
1. Graph-Aware Feedback Links
Standard DFA uses random matrices to project errors. DFA-GNN improves upon this by incorporating the graph's adjacency matrix into the feedback path. This ensures that the error signals propagated to hidden layers are informed by the relational structure of the nodes, maintaining the "graph-aware" nature of the learning process.
2. Pseudo Error Generator
In semi-supervised learning, errors can only be calculated for the small subset of nodes with known labels. To provide training signals for the rest of the graph, DFA-GNN employs a "label spreading" technique. It takes the known residual errors from training nodes and diffuses them across edges to estimate "pseudo-errors" for unlabeled nodes.
3. Node Filtering Mask
To prevent the model from learning from low-quality estimations, a filtering mask is applied. This mask uses a threshold ($\epsilon$) to exclude nodes with high uncertainty, ensuring that only reliable pseudo-errors drive the weight updates.
4. Parallel Layer Updates
Because error signals are projected directly from the output to each layer simultaneously, the requirement for a sequential chain of derivatives is removed. This allows all layers in the GNN to update their parameters in parallel, breaking the "update locking" inherent in BP.
Experimental Results
The researchers evaluated DFA-GNN across 10 diverse benchmarks, including citation networks (Cora, CiteSeer), co-purchase graphs (Amazon), and large-scale datasets (Reddit, ogbn-arxiv).
Accuracy and Performance
DFA-GNN demonstrated superior performance, outperforming standard BP on 7 out of 10 benchmarks. On the Cora dataset, DFA-GNN achieved 87.72% accuracy, surpassing BP's 86.04%. More impressively, it vastly outperformed other non-BP baselines like PEPITA and Forward-Forward (FF), which often failed to reach 50% accuracy on complex graph tasks.
Robustness and Scalability
- Over-smoothing Resistance: As GNNs grow deeper, node representations often become indistinguishable (over-smoothing). DFA-GNN mitigates this because the global loss directly supervises every layer, maintaining distinct features.
- Structural Resilience: The framework is remarkably robust against structural noise, such as the random addition or removal of edges. The pseudo-error mechanism effectively acts as a regularizer.
- Large-Scale Utility: On the Reddit dataset, DFA-GNN maintained a high accuracy of 94.49% with a manageable memory footprint, proving its viability for real-world, large-scale applications.
Discussion and Future Work
DFA-GNN proves that forward-learning is not merely a biological curiosity but a competitive alternative to BP for graph-based data. Mathematically, the authors demonstrate that forward weights eventually align with the fixed random feedback weights to create valid descent directions.
However, some challenges remain. The current implementation of pseudo-error generation and filtering increases the training time per epoch compared to standard BP. Additionally, the model introduces new hyperparameters, such as the filtering threshold $\epsilon$ and smoothing factor $\alpha$, which require careful tuning. Future research will focus on optimizing these overheads and extending the DFA-GNN framework to more complex architectures like heterogeneous Graph Transformers.
Summary updated