Abstract
Graph representation learning is gaining popularity in a wide range of applications, such as social networks analysis, computational biology, and recommender systems. However, different with positive results from many academic studies, applying graph neural networks (GNNs) in a real-world application is still challenging due to non-stationary environments. The underlying distribution of streaming data changes unexpectedly, resulting in different graph structures (a.k.a., concept drift). Therefore, it is essential to devise a robust graph learning technique so that the model does not overfit to the training graphs. In this work, we present Hop Sampling, a straightforward regularization method that can effectively prevent GNNs from overfitting. The hop sampling randomly selects the number of propagation steps rather than fixing it, and by doing so, it encourages the model to learn meaningful node representation for all intermediate propagation layers and to experience a variety of plausible graphs that are not in the training set. Particularly, we describe the use case of our method in recommender systems, a representative example of the real-world non-stationary case. We evaluated hop sampling on a large-scale real-world LINE dataset and conducted an online A/B/n test in LINE Coupon recommender systems of LINE Wallet Tab. Experimental results demonstrate that the proposed scheme improves the prediction accuracy of GNNs. We observed hop sampling provides 7.97% and 16.93% improvements for NDCG and MAP compared to non-regularized GNN models in our online service. Furthermore, models using hop sampling alleviate the oversmoothing issue in GNNs enabling a deeper model as well as more diversified representation.
Introduction
In academic settings, Graph Neural Networks (GNNs) are often evaluated on static datasets with fixed structures. However, real-world applications—such as the LINE Coupon service—operate in non-stationary environments. In these settings, new users and items appear daily, and relationships shift rapidly, a phenomenon known as "concept drift." Standard GNNs tend to overfit the specific connectivity of the training graph, leading to performance drops when the live graph at inference time differs. Additionally, GNNs suffer from oversmoothing, where node representations converge and become indistinguishable as more layers (hops) are added, destroying the model's ability to provide personalized recommendations.
The Hop Sampling Method
Hop Sampling (HS) is a simple yet effective regularization technique that can be integrated into existing GNN architectures like GCN or APPNP.
- Standard Approach: Researchers typically choose a fixed number of propagation steps ($K$). Every node aggregates information from its $K$-hop neighborhood in every training batch.
- Hop Sampling Approach: During training, for each batch, the algorithm randomly selects a hop count $k$ from a discrete uniform distribution $\mathcal{U}(0, K)$.
By introducing uncertainty regarding the number of hops, the model is forced to optimize for the expectation of the embedding rather than a single fixed depth. This acts as a form of "depth-wise dropout," ensuring the model learns useful features at every intermediate stage and experiences a wider variety of plausible graph structures. During inference, sampling is disabled, and the model uses the full $K$ steps.
Experimental Evaluation
Dataset and Setup
The study utilized a large-scale dataset from the LINE Coupon Service, featuring 120,000 users and 517 items in a bipartite graph. The environment is highly non-stationary; between consecutive days, roughly 94% of nodes and 97% of edges are new. This makes it an ideal stress test for robust graph learning.
Robustness to Oversmoothing
A key challenge in GNNs is the performance collapse as the number of propagation steps increases. As shown in the figure below, standard models like GCN experience a sharp decline in accuracy (NDCG@10) as depth increases beyond a few hops. In contrast, models using Hop Sampling (APPNP-HS and GCN-HS) maintain high and stable performance even up to 16 hops, effectively mitigating the oversmoothing problem.
Preventing Overfitting in Non-Stationary Environments
Hop Sampling also serves as a powerful regularizer against overfitting to specific temporal snapshots of the graph. In the following figure, we observe that while the standard APPNP model's performance degrades significantly as training progresses (likely due to overfitting the training graph's specific structure), the APPNP-HS model maintains consistent performance across epochs.
Key Results and Conclusion
Offline evaluations on the LINE dataset showed that APPNP-HS achieved an NDCG@10 of 0.3039, outperforming standard APPNP (0.2919) and GCN (0.2114). Furthermore, Hop Sampling naturally improved recommendation diversity, as measured by Inter-List Diversity (ILD), by preventing user representations from smoothing into a single average.
In live A/B tests on the LINE Wallet Tab, Hop Sampling models provided a 7.97% improvement in NDCG and a 16.93% improvement in MAP over the production baseline. These results demonstrate that Hop Sampling is a simple, computationally inexpensive, and highly effective method for deploying robust GNNs in dynamic, real-world environments.
Summary updated