Abstract
Recently, recommender systems based on Graph Convolution Network (GCN) have become a research hotspot, especially in collaborative filtering. However, most GCN-based models have inferior embedding propagation mechanism, leading to low information extraction efficiency. Besides, the existing methods suffer from high computational complexity for large user-item interaction graphs. In order to solve the above problems, we propose LII-GCCF that integrates Linear transformation, Initial residual and Identity mapping into the Graph Convolutional Collaborative Filtering model. First, initial residual and identity mapping are applied to optimize the information propagation of graph convolution, which provide abundant interaction and alleviate information loss problem. Second, LII-GCCF removes the unnecessary nonlinear transformation based on the characteristics of collaborative filtering to simplify the graph convolution process. Comprehensive experiments are conducted on two public datasets, and the results demonstrate that LII-GCCF has a significant improvement over other state-of-the-art methods.
What Problem Does This Paper Address?
Collaborative Filtering (CF) is the cornerstone of modern recommendation engines, predicting user preferences by leveraging historical interactions. Recent advancements have utilized Graph Convolutional Networks (GCNs) to capture "high-order connectivity"—the structural relationships where, for example, User A might prefer items liked by User B if they share common interests.
However, standard GCN architectures, originally designed for feature-rich domains like social networks or citation graphs, face significant hurdles in CF. In recommendation tasks, nodes are typically unique identifiers (IDs) mapped to latent embeddings rather than nodes with rich semantic attributes. The authors identify two critical flaws in applying standard GCNs to this domain:
- Low Information Extraction Efficiency: Deep GCNs often suffer from "over-smoothing," a phenomenon where node embeddings become increasingly similar and lose their discriminative power after multiple layers of propagation.
- Redundant Complexity: Standard GCNs rely on non-linear activation functions (e.g., ReLU). In the context of ID-based embeddings, these operations are computationally expensive and often fail to provide meaningful feature extraction, leading to unnecessary overhead.
Methods
The proposed model, LII-GCCF, introduces a streamlined architecture based on three core principles: Linear transformation, Initial residual, and Identity mapping.
The architecture simplifies the graph convolution process through the following components:
- Initial Residual: To mitigate over-smoothing, LII-GCCF incorporates a "shortcut" connection from the initial embedding layer to every subsequent layer. This ensures that the unique identity of each user and item is preserved throughout the deep propagation process.
- Identity Mapping: By adding an identity matrix to the weight matrix at each layer, the model is encouraged to retain existing useful information while selectively integrating new signals from neighboring nodes. This makes the training process more stable and robust.
- Linear Transformation: Crucially, the authors remove non-linear activation functions between layers. They argue that since initial ID embeddings lack inherent semantic meaning, non-linearities do not aid in feature extraction but instead introduce noise and computational lag.
Experimental Setup
The model was rigorously evaluated on two benchmark datasets:
- Gowalla: A location-based social network dataset featuring user check-ins.
- Amazon-Book: A large-scale dataset comprising product reviews and ratings within the book category.
Both datasets underwent pre-processing to filter out inactive users and items (those with fewer than 10 interactions). The data was partitioned into training (80%), validation (10%), and test (10%) sets for robust performance measurement.
Key Insights and Ideas
- The "ID Paradox": While deep learning typically thrives on non-linearity, collaborative filtering presents a unique case. Because the input consists of arbitrary ID vectors, the primary goal is the propagation of signals across the graph rather than the transformation of complex features. In this environment, linear models can be more expressive and significantly easier to optimize.
- Optimal Depth: Despite the inclusion of residuals, the authors found that a shallow architecture—specifically two layers—yielded the best performance. Beyond this depth, the noise introduced by distant neighbors begins to obscure the relevant signals.
- Computational Efficiency: By stripping away non-linearities, LII-GCCF achieves superior speed. On the Gowalla dataset, it required approximately 80 seconds per epoch, a significant improvement over the 115 seconds required by the state-of-the-art NGCF model, all while utilizing fewer parameters.
Key Results
LII-GCCF demonstrated substantial improvements over established baselines, including Matrix Factorization (MF), PinSage, and Neural Graph Collaborative Filtering (NGCF).
- Accuracy Gains: On Gowalla, LII-GCCF improved the Hit Ratio (HR@20) by 8.22% and the Normalized Discounted Cumulative Gain (NDCG@20) by 9.65% compared to the strongest baseline.
- Amazon-Book Performance: The model showed even more pronounced gains on the larger Amazon dataset, achieving a 17.44% improvement in HR@20.
- Ablation Studies: The authors confirmed that each "LII" component is vital; removing either the initial residual or the identity mapping resulted in a measurable decline in recommendation quality.
Conclusion and Future Work
LII-GCCF demonstrates that a "less is more" approach can be highly effective for GCN-based recommendation. By simplifying the convolution process and focusing on efficient signal propagation, the model achieves state-of-the-art results with reduced complexity.
Future research directions include exploring more sophisticated interaction functions beyond the simple inner product and extending these simplified GCN principles to more complex graph structures, such as knowledge graphs or heterogeneous social networks. Additionally, addressing the model's sensitivity to hyperparameters ($\alpha$ and $\beta$) remains an area for further optimization.
Summary updated