Abstract
The flexibility and effectiveness of message passing based graph neural networks (GNNs) induced considerable advances in deep learning on graph-structured data. In such approaches, GNNs recursively update node representations based on their neighbors and they gain expressivity through the use of node and edge attribute vectors. E.g., in computational tasks such as physics and chemistry usage of edge attributes such as relative position or distance proved to be essential. In this work, we address not what kind of attributes to use, but how to condition on this information to improve model performance. We consider three types of conditioning; weak, strong, and pure, which respectively relate to concatenation-based conditioning, gating, and transformations that are causally dependent on the attributes. This categorization provides a unifying viewpoint on different classes of GNNs, from separable convolutions to various forms of message passing networks. We provide an empirical study on the effect of conditioning methods in several tasks in computational chemistry.
Introduction
In the development of Graph Neural Networks (GNNs), significant research focus is placed on feature engineering—determining which attributes, such as pairwise distances or bond angles, should be included. However, the mechanism by which these models ingest such information is often overlooked. Most state-of-the-art GNNs in chemistry and physics, such as EGNN or SchNet, rely on message-passing conditioned on geometric attributes. This paper formalizes and benchmarks different "conditioning" methods, demonstrating that the choice of conditioning mechanism significantly impacts model performance and efficiency.
Categorizing Conditioning Methods
The authors propose a hierarchy of conditioning based on the degree of dependency between the transformation and the attribute ($a$):
- Weak Conditioning: The attribute $a$ is concatenated with the hidden features $h$. This treats the attribute as an adaptive bias in a standard linear layer.
- Strong Conditioning: The attribute is used to gate the features. It passes through its own linear layer to create a scaling vector, which is then element-wise multiplied by the transformed features. This approach is mathematically related to depth-wise separable convolutions.
- Pure Conditioning: The attribute causally parametrizes the weight matrix itself. While theoretically the most direct form of conditioning, it is computationally intensive.
The study also investigates Deep Conditioning, where these methods are applied across all layers of a message-passing function rather than just the initial layer.
Empirical Evaluation
The researchers evaluated these methods on two primary benchmarks in computational chemistry: QM9 (predicting quantum properties of ~134k small molecules) and MD17 (predicting conformational energies from molecular dynamics trajectories).
Key Insights
- The Gating Advantage: Strong conditioning forces the model to utilize the attribute. Unlike weak conditioning, where the model might learn to ignore the attribute-dependent bias, strong conditioning uses a gate that can "kill" the signal if necessary, creating a more robust inductive bias.
- Efficiency vs. Expressivity: While "Pure" conditioning is highly expressive, it suffers from a massive computational bottleneck. Training one epoch of a "Pure-EGNN" was found to be 267x slower than the "Strong" variant.
- Network Depth: The performance gap between weak and strong conditioning is most pronounced in shallow networks, suggesting that strong conditioning is a more parameter-efficient way to incorporate geometric information. Furthermore, "deep conditioning" (applying gating at every layer) consistently outperformed shallow conditioning.
Results
On the QM9 dataset, transitioning from weak to strong conditioning in an EGNN architecture resulted in an average Mean Absolute Error (MAE) improvement of 10% across 12 molecular properties.
In MD17 experiments, strong conditioning improved energy prediction for the majority of molecules, yielding an average performance gain of 14.5%.
Conclusion
Strong conditioning (gating) emerges as the optimal balance between performance and computational efficiency. It provides nearly all the benefits of the more complex "Pure" conditioning while maintaining the speed of the "Weak" baseline. These findings suggest that future geometric GNN architectures should prioritize "strong" deep conditioning to maximize the utility of the geometric data provided.
Summary updated