Many graphs are directed: a paper cites another, money moves from one account to another, one page links to another. On these graphs, keeping the arrows matters a great deal. On the heterophilic Roman-Empire benchmark, a graph where linked nodes usually belong to different classes, an ordinary graph convolutional network reaches 56% accuracy. Treating incoming and outgoing edges separately raises that to about 74% (Edge Directionality Improves Learning on Heterophilic Graphs). The models that combine direction with long-range context are expensive, though. Multi-scale PageRank convolutions run out of memory at 50 million nodes (Digraph Inception Convolutional Networks). Complex-valued "magnetic" operators cost roughly four times as much as real-valued ones (MagNet: A Neural Network for Directed Graphs). Directed transformers carry quadratic attention costs (Directed Graph Transformers). In many deployments, such as a new account joining a transaction graph or a new paper entering a citation index, the model at inference time can only afford a node's own features and its immediate in- and out-neighbors.
We propose to train such a one-hop student by knowledge distillation, meaning the student is trained to mimic a large "teacher" model rather than only the hard labels. The new part is that we distill direction-separated long-range targets. The student has separate incoming and outgoing channels. We train its incoming channel to predict what the teacher believes about the node's multi-hop ancestors, and its outgoing channel to predict what the teacher believes about the node's multi-hop descendants. The bet is that ordinary output-matching distillation lets the student blur the two directions together. Splitting the targets by direction keeps the asymmetry that makes direction useful on heterophilic graphs, and it does so without any multi-hop computation at inference.
Core Claim
A one-hop directed student can recover most of a deep directed teacher's accuracy on direction-sensitive heterophilic graphs under three conditions. The student must receive the teacher's knowledge in two separate parts: an upstream summary aggregated over predecessors and a downstream summary aggregated over successors. Each part must be routed into the matching student channel. The student must also receive the usual soft-label loss. Matching the teacher's output distribution alone (logit-only distillation) will leave a measurable gap, and that gap will be largest exactly where the Random Flip Test shows the task depends on direction.
Formally, let $p_T(v)$ be the teacher's softened class distribution at node $v$. Let $P_{\text{in}}$ and $P_{\text{out}}$ be the row-normalized random-walk matrices along reversed and forward edges. Let $h^{\text{in}}_v$ and $h^{\text{out}}_v$ be the student's one-hop incoming and outgoing channel states, with small linear readouts $g_{\text{in}}$ and $g_{\text{out}}$. The upstream and downstream targets are
$$ u^{\text{in}}_v=\sum_{k=1}^{K}\beta^k\big(P_{\text{in}}^k p_T\big)_v,\qquad u^{\text{out}}_v=\sum_{k=1}^{K}\beta^k\big(P_{\text{out}}^k p_T\big)_v , $$
where $K$ is the horizon and $\beta\in(0,1)$ is a decay. The student minimizes
$$ \mathcal{L}=\mathrm{CE}(y_v,p_S(v))+\lambda\,\mathrm{KL}\big(p_T(v)\,\|\,p_S(v)\big)+\mu\Big[\mathrm{KL}\big(\hat u^{\text{in}}_v\,\|\,g_{\text{in}}(h^{\text{in}}_v)\big)+\mathrm{KL}\big(\hat u^{\text{out}}_v\,\|\,g_{\text{out}}(h^{\text{out}}_v)\big)\Big]. $$
Here $p_S(v)$ is the student's prediction, $y_v$ the label, $\hat u$ the target renormalized to a distribution, and $\lambda,\mu$ loss weights. The targets are computed once, offline, from the teacher. The student never touches $P^k$ at inference.
Positioning
Two closely related works each cover two of our three ingredients. Frameless Graph Knowledge Distillation distills multi-scale framelet teachers into MLP students, using learnable score vectors that balance features against structure. Its framelets are undirected, so it inherits the symmetrization collapse that Dir-GNN documents. We reuse its idea that multi-scale structure can be compressed into a cheap student, but we make the scales directional and route them into matching student channels. Toward Data-centric Directed Graph Learning: An Entropy-driven Approach (EDEN) does distill on directed graphs. Its teacher is a hierarchy tree built from the data, and its students are full directed GNN backbones, so the students' receptive field is never constrained. Our teacher is a long-range directed network, our student is deliberately limited to one hop, and our question is what directional knowledge survives that compression.
The architectures we use as teachers supply the method. These are DiGCN's multi-scale directed proximities, MagNet's phase-encoded direction, and DiGT's dual sender/receiver roles. DAGformer: Directed Acyclic Graph Transformer restricts attention to ancestors and descendants, which is close in spirit to our upstream/downstream split. It requires acyclic graphs, however, and our random-walk targets do not. Sink nodes break naive forward walks, as discussed in Disobeying Directions: Switching Random Walk Filters for Unsupervised Node Embedding Learning on Directed Graphs, so we add self-loops in $P_{\text{out}}$ at sinks and report how often this fires.
The geometric intuition comes from CCNet: Criss-Cross Attention for Semantic Segmentation. CCNet gets global image context by decomposing it along two axes, horizontal and vertical. On a digraph the natural axes are upstream and downstream, and we decompose the teacher's context along those. Finally, A Unified Knowledge Distillation Framework for Deep Directed Graphical Models acts as a caution. It shows that output-only distillation of deep directed hierarchies drifts, and that intermediate (latent) matching fixes this. On CelebA, its distilled student reaches a quality score (FID, lower is better) of 5.38, against 6.23 for layer-local distillation and 6.10 for training from scratch. That result motivates our intermediate channel targets. Its setting is stochastic latent layers, not graph hops, so we treat it as motivation rather than evidence.
Research Program
We first train teachers on each benchmark: a multi-layer Dir-GNN, DiGCN, and MagNet, plus DiGT where it fits in memory. From each teacher we compute $u^{\text{in}}$ and $u^{\text{out}}$ for $K\in\{2,4,8\}$.
The student is a single Dir-GNN layer. It has separate mean aggregation over in-neighbors and out-neighbors, a self branch, and an MLP head. We also include a pure-MLP student so the comparison links back to FGD.
The analysis has three parts. First, we measure how the distillation gap (teacher accuracy minus student accuracy) changes with teacher horizon $K$, separately for homophilic and heterophilic graphs. Second, we test whether the student's channels actually specialize. We probe $h^{\text{in}}$ for $u^{\text{out}}$ and the reverse, and we expect cross-prediction to be poor if specialization occurs. Third, we check whether the student inherits the teacher's sensitivity to direction. We apply DiGT's Random Flip Test to teacher and student and compare their accuracy curves as the fraction of flipped edges grows. A student that matches the teacher's accuracy but is flat under flips has learned something other than direction.
Baselines
The naive reference point is the same one-hop student trained from scratch on hard labels. There is no reported result on Lacuna for a one-hop Dir-GNN on Roman-Empire. We therefore run it ourselves with the same tuning budget as every other student: an identical hyperparameter search size and the same seeds and splits.
The strongest existing method is the full-capacity directed teacher. On Roman-Empire, the file reports Dir-GCN at about 74% accuracy against 56% for GCN (Edge Directionality Improves Learning on Heterophilic Graphs). EDEN reports average node-level gains of 3.12% over the best baselines across its 14 directed datasets, but no absolute per-dataset numbers are given in the file. Both serve as ceilings rather than competitors at equal cost. We also report inference latency and memory for each. Our student counts as successful if it closes at least half of the gap between the scratch student and the teacher on Roman-Empire, Chameleon, Squirrel, and the WebKB graphs, while costing one sparse pass.
The strongest same-cost competitor is logit-only distillation, the GLNN-style soft-label matching that FGD builds on. We give it the identical one-hop directed student, the same teacher, and the same search budget. The key ablations isolate our ingredient. One setting keeps the channel targets but swaps them, so the incoming channel predicts downstream and the outgoing channel predicts upstream. Another setting replaces both targets with a single symmetrized target fed to both channels. If the swapped or symmetrized versions match the aligned version, then direction-aligned routing is not the cause of any gain. A further control trains the teacher on the symmetrized graph, to check that the knowledge being transferred is directional at all.
Why not just…?
Why not just precompute directed multi-hop features and train an MLP on them? Concatenating $P_{\text{in}}^kX$ and $P_{\text{out}}^kX$ and training an MLP is the most serious threat. On transductive benchmarks it may match us. Its weakness is the deployment regime we target. For a newly arriving node it needs a $K$-hop neighborhood fetch at inference, which is exactly the cost we rule out. We will run it anyway as a cost-matched-at-training, more-expensive-at-inference reference. We will also run an inductive split where held-out nodes arrive with only their one-hop edges. If precomputation still wins inductively, distillation is unnecessary and we will say so.
Why not just use plain logit distillation into the directed student? This may be enough on graphs where the Random Flip Test shows direction barely matters. Our claim is narrower: aligned channel targets should beat logit-only distillation specifically on direction-sensitive graphs. The head-to-head comparison stratified by flip sensitivity decides this, and a null result there falsifies the proposal.
Why not just symmetrize and distill with undirected FGD? The Roman-Empire numbers already show that symmetrization costs about 18 points at the architecture level. Explanations also become unfaithful when direction is ignored (Ignoring Directionality Leads to Compromised Graph Neural Network Explanations). We include an FGD-style undirected student so that this cost can be measured under distillation rather than assumed.
Proposed Experiments
We run node classification on Roman-Empire, Chameleon, Squirrel, Cornell, Texas, Wisconsin, Tolokers, Cora-ML, CiteSeer, and ogbn-arxiv, plus Twitter5 and FlowGraph6 from DiGT, where DiGT beats standard graph transformers by over 7% and by 3.06% respectively. We report accuracy over ten seeds, inference latency per node, and peak memory. Each dataset is first screened with the Random Flip Test so results can be grouped by how much the task depends on direction. We also include MNIST/CIFAR-10 graph versions, which DiGT reports as flip-invariant, as negative controls: on those, our method should show no advantage over logit-only distillation.
Three checks decide the outcome. First, on flip-sensitive heterophilic graphs, the aligned student must beat logit-only distillation and both the swapped and symmetrized ablations. Second, the student's accuracy curve under edge flips must follow the teacher's rather than stay flat. Third, in the inductive one-hop setting, the aligned student must match or beat the precomputed directed-feature MLP while using strictly less inference work. Failing the first check means direction-split targets add nothing. Failing the third means the practical case for the method rests on transductive settings alone.
References
- Edge Directionality Improves Learning on Heterophilic Graphs
- Digraph Inception Convolutional Networks
- MagNet: A Neural Network for Directed Graphs
- Directed Graph Transformers
- Frameless Graph Knowledge Distillation
- Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
- DAGformer: Directed Acyclic Graph Transformer
- Disobeying Directions: Switching Random Walk Filters for Unsupervised Node Embedding Learning on Directed Graphs
- CCNet: Criss-Cross Attention for Semantic Segmentation
- A Unified Knowledge Distillation Framework for Deep Directed Graphical Models
- Ignoring Directionality Leads to Compromised Graph Neural Network Explanations