Preserving Dirichlet Energy in Deep Graph Neural Networks

This direction is merged into another direction: graph over-smoothing (over-smoothing), Dirichlet energy preservation. Its last version stays readable and citable here.

Version 1 of 151 papers73 conceptsfrozen 2026-10-04

In a standard Graph Neural Network (GNN), adding layers is a double-edged sword. While deeper architectures should theoretically capture more complex, long-range dependencies, they often fall victim to over-smoothing: a phenomenon where node features converge to a uniform value, making them indistinguishable.

To understand why this happens and how to fix it, researchers have turned to a concept from mathematical physics: Dirichlet energy. By treating GNNs not just as black-box encoders but as physical systems evolving over time, we can design architectures that maintain feature diversity even at extreme depths.

The Metric of Smoothness: Dirichlet Energy

Intuitively, Dirichlet energy measures how much node features vary across the edges of a graph. If every node has the exact same representation, the energy is zero. If neighboring nodes have very different representations, the energy is high.

In A Note on Over-Smoothing for Graph Neural Networks, Chen Cai and Yusu Wang prove that in many deep GNNs, this energy decays exponentially with depth. Standard message passing acts like a heat diffusion process; just as heat eventually spreads evenly across a metal plate, information in a GNN eventually "blurs" across the graph until the signal is lost. T. Konstantin Rusch and Michael M. Bronstein argue in their survey that maintaining this energy is a necessary condition for expressivity in deep models.

From Diffusion to Reaction-Diffusion

If standard GNNs are purely "diffusive" (smoothing things out), the most direct solution is to add a "reaction" term that pushes back. This is the core idea behind Reaction-Diffusion Networks.

In ACMP: Allen-Cahn Message Passing, Yuelin Wang and colleagues borrow the Allen-Cahn equation from materials science. They introduce a "double-well potential" that forces node features toward specific values, effectively creating a repulsive force that counteracts the smoothing effect of neighbors. Similarly, Jeongwhan Choi introduced GREAD, which uses learnable reaction terms to ensure that Dirichlet energy stays healthy across dozens of layers. These models don't just slow down smoothing; they actively "sharpen" the signal.

The Spectral Solution: High-Pass Filtering

Another way to look at over-smoothing is through the lens of signal processing. Graph convolution is typically a low-pass filter, meaning it keeps the "global trend" (smoothness) and throws away the "local details" (high-frequency differences).

To combat this, Xinliang Liu and Yu Guang Wang proposed Framelet Message Passing (FMP). By decomposing the graph signal into low-pass and high-pass components, FMP explicitly tracks and propagates the high-frequency details that traditional GNNs discard. This ensures that the Dirichlet energy—which is largely contained in those high-frequency components—is conserved. Philipp Grohs and Helmut Bölcskei provided the theoretical backbone for this in their work on Energy Propagation, showing that analytic filters can guarantee energy conservation regardless of depth.

Constraining the Energy "Goldilocks Zone"

Rather than changing the physics of the message passing, some researchers suggest simply "policing" the energy levels.

In Dirichlet Energy Constrained Learning, Kaixiong Zhou and Xiao Huang introduced EGNN. This framework treats Dirichlet energy as a mathematical constraint during training. By forcing the energy to stay within a specific range—neither collapsing to zero (over-smoothing) nor exploding into noise—they successfully trained GNNs up to 64 layers deep.

Other structural interventions include:

Why It Works: The Intuition of Distinctness

The shift toward energy-based GNNs represents a move from "averaging" to "transporting." While early GNNs were designed to find similarities between neighbors (homophily), modern tasks—especially in chemistry or fraud detection—require the model to spot subtle differences (heterophily).

By prioritizing the preservation of Dirichlet energy, these new architectures ensure that every layer of the network has something unique to say. Whether through the repulsive forces of the Allen-Cahn equation or the multiscale views of Framelets, the goal is the same: to keep the "colors" of the graph from mixing into a single, useless shade of gray.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_preserving_dirichlet_energy_in_deep_grap_1c2c24c4,
  title = {Preserving Dirichlet Energy in Deep Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preserving-dirichlet-energy-in-deep-graph-neural-networks/rdr_1c2c24c46051440c8d313bf73f07b9ac},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_preserving_dirichlet_energy_in_deep_grap_1c2c24c4_v1,
  title = {Preserving Dirichlet Energy in Deep Graph Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/preserving-dirichlet-energy-in-deep-graph-neural-networks/rdr_1c2c24c46051440c8d313bf73f07b9ac/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}