Abstract
Graph Neural Networks (GNNs) have become a cornerstone of multimedia recommendation systems. However, they often suffer from "over-smoothing," where user representations converge toward global popularity patterns at the expense of unique individual preferences. This paper proposes Principal Graph Learning (PGL), a framework designed to capture high-frequency local structural features—the "individual information"—that standard GNNs typically filter out. By extracting and learning from a principal subgraph, PGL achieves a 9% average performance improvement over state-of-the-art models across multiple Amazon datasets.
Introduction: The Over-Smoothing Challenge
In multimedia recommendation, GNNs leverage user-item interaction graphs to aggregate neighbor information via message passing. While effective at capturing global co-occurrence patterns (low-frequency signals), these models act as low-pass filters that suppress high-frequency signals. These suppressed signals often contain critical "individual information"—the unique, non-conforming tastes of specific users.
When models focus exclusively on global trends, user embeddings become excessively similar, leading to a lack of uniformity in the vector space. This makes it difficult for systems to distinguish between users with nuanced differences in taste. PGL addresses this by mining the local structural features of the graph to preserve distinct user identities.
Principal Graph Learning (PGL)
The PGL framework shifts the focus from the entire noisy interaction graph to a "principal subgraph" during the training phase. This subgraph is designed to retain the most informative structural components while discarding noise.
Subgraph Extraction Strategies
PGL employs two primary operators for subgraph extraction:
- Global-Aware Extraction: Utilizes randomized Singular Value Decomposition (SVD) to approximate the most significant global structural components of the graph.
- Local-Aware Extraction: Employs a multinomial sampling strategy based on node exposure. This approach prioritizes edges connected to less-popular items, as these interactions are often more indicative of specific individual preferences than interactions with "blockbuster" items.
During training, the model performs message passing on this principal subgraph and incorporates a Self-Supervised Learning (SSL) task to further enhance embedding distinguishability. For inference, the model reverts to the complete graph to ensure no global context is lost.
Experimental Evaluation
The researchers evaluated PGL using three categories from the Amazon dataset: Baby, Sports and Outdoors, and Clothing, Shoes, and Jewelry. These datasets provide rich multimedia context, including 4,096-dimensional visual features from images and 384-dimensional textual features from product descriptions.
Key Results and Insights
PGL consistently outperformed 14 state-of-the-art baselines, including LightGCN, LATTICE, and FREEDOM. On average, PGL delivered a 9% improvement in Recall@20 and NDCG@20 metrics.
Embedding Uniformity
A key finding of the study is the correlation between embedding uniformity and recommendation quality. Visualizing the embeddings on a unit circle reveals that PGL produces the most spread-out and distinguishable representations.
By effectively capturing high-frequency signals, PGL ensures that user profiles remain distinct. Furthermore, the use of a sparse principal subgraph during training makes PGL more computationally efficient than complex baselines like LATTICE, despite its superior performance.
Conclusion
Principal Graph Learning offers a robust solution to the over-smoothing problem in GNN-based multimedia recommendation. By "minding individual information" through targeted subgraph extraction, PGL balances global context with local nuance. Future research will explore more sophisticated local feature mining and the scalability of PGL to massive, industrial-scale graphs.
Summary updated