Discovery Science

Clemens Damke, Eyke HüllermeierLecture Notes in Computer Science 2021 · DOI · arXiv · OpenAlex

Abstract

In scientific domains such as drug discovery, the primary objective is often to identify the most promising candidates from a large pool of structured objects. Traditional methods rely on point-wise regression to predict absolute properties, which can be sensitive to outliers and noise. This paper introduces RankGNN, a framework that integrates Graph Neural Networks (GNNs) with neural "Learning to Rank" (LtR) techniques. By training on pair-wise preferences rather than absolute values, RankGNN focuses on the underlying order of objects. Experimental results across molecular and synthetic datasets demonstrate that RankGNN significantly outperforms regression-based baselines in ranking accuracy and robustness.

Introduction

The ability to rank structured objects is critical in fields like chemistry and materials science. For instance, in drug discovery, experts must prioritize thousands of candidate molecules for experimental validation. Historically, this task has been approached through manual feature engineering or point-wise regression. In the latter, a model is trained to predict a specific scalar value (e.g., solubility) for each graph, and the graphs are subsequently sorted.

However, point-wise regression is fundamentally limited. It aims to minimize the distance to specific numerical targets (e.g., Mean Squared Error), making it highly susceptible to distortion by outliers. Furthermore, the absolute scale of the data is often less important than the relative order. To address these issues, the authors propose a pair-wise approach: teaching a model to compare two graphs and determine which is "better." This shifts the learning objective from absolute value estimation to preference learning, which is more aligned with the ultimate goal of discovery.

Methods

The RankGNN framework consists of two primary modules: a GNN-based feature extractor and a Comparator Network.

The GNN Component

The GNN acts as a feature extractor, mapping a graph's nodes and edges into a fixed-length embedding vector. The authors evaluated several architectures, including Graph Convolutional Networks (GCN), Graph Isomorphism Networks (GIN), and the more expressive 2-WL-GNN. The choice of GNN is crucial, as certain architectures are structurally limited in their ability to detect specific graph patterns, such as cycles.

The Comparator Network

The comparator network receives two graph embeddings and predicts a preference. Two specific architectures were evaluated:

  • DirectRanker: This model is designed to enforce mathematical properties such as transitivity (i.e., if A > B and B > C, then A > C).
  • CmpNN: A more flexible architecture capable of representing non-transitive preferences, although transitivity is typically preferred for scientific ranking tasks.
The RankGNN architecture for pair-wise graph ranking.

Technical Innovation: Efficient Batching

To ensure computational efficiency, the authors implemented a smart batching strategy. Instead of redundantly passing the same graph through the GNN for every pair it appears in, the model encodes a batch of unique graphs once. A lookup table is then used to retrieve these embeddings and form the necessary pairs for the ranking loss calculation. This significantly reduces the computational overhead associated with pair-wise training.

Experimental Setup

Due to a lack of dedicated graph ranking benchmarks, the authors adapted five existing graph regression datasets for ranking tasks:

  1. TRIANGLES: A synthetic dataset where graphs are ranked by the number of 3-cliques (triangles) they contain.
  2. OGB-molesol, -mollipo, and -molfreesolv: Molecular datasets from the Open Graph Benchmark focused on solubility and lipophilicity.
  3. ZINC: A large-scale dataset containing 250,000 commercially available chemicals.

Key Insights and Results

The performance of the models was evaluated using Kendall’s $\tau_B$, a correlation coefficient where 1.0 indicates a perfect ranking.

Rank Loss vs. MSE

A key finding is that pair-wise rank loss (binary cross-entropy on preferences) is inherently more robust than point-wise MSE loss. While MSE can be heavily influenced by extreme outliers, the ranking loss only requires that the outlier be correctly positioned relative to other items. This leads to more stable and accurate rankings.

Structural Expressivity

The TRIANGLES dataset highlighted the importance of GNN expressivity. Standard GNNs like GCN and GIN failed to perfectly rank graphs by triangle count because they are structurally "blind" to cycles. In contrast, the higher-order 2-WL-GNN achieved a perfect 1.0 score, demonstrating that ranking performance is bounded by the GNN's ability to extract relevant structural features.

Performance on Molecular Data

On the ZINC and OGB-mollipo datasets, RankGNN variants consistently outperformed regression baselines. For example, on the ZINC dataset, the best RankGNN achieved a $\tau_B$ of approximately 0.89, whereas standard utility regression scores ranged from 0.55 to 0.80. Visualizations of the learned "utility functions" showed that RankGNNs effectively "stretch" the utility scale, making the ordering clearer and more linear than the original noisy regression targets.

Conclusion and Future Work

RankGNN demonstrates the power of combining GNNs with pair-wise ranking objectives for structured data. While the current study used regression targets to simulate ranking data, future work should focus on real-world datasets where ground truth is derived from human expert preferences. Additionally, exploring "list-wise" ranking methods—which consider an entire set of objects simultaneously—could capture more complex, context-dependent preferences. Finally, the neural GNN-based approach proved more scalable than kernel-based methods like RankSVM, making it a practical choice for large-scale scientific discovery.

Summary updated

BibTeX
@inproceedings{damke2021discovery,
  author = {Clemens Damke and Eyke Hüllermeier},
  title = {{Discovery Science}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2021},
  doi = {10.1007/978-3-030-88942-5},
}
Figure