Abstract
Multimodal Recommender Systems (MRSs) aim to enhance suggestions by integrating product images and textual descriptions. However, current models often suffer from an overreliance on text, low information density in raw visual features, and a failure to capture complex multi-way relationships. We propose DOGE (LLMs-EnhanceD Hyper-KnOwledge Graph REcommender), which utilizes Multimodal Large Language Models (MLLMs) to synthesize a "semantic modality" that bridges the gap between visual and textual data. By structuring these enriched features into a Hyper-Knowledge Graph (HKG), DOGE models high-order relational dependencies. Our evaluation on real-world datasets shows that DOGE achieves a significant 7.2% performance boost over state-of-the-art baselines.
Introduction
Modern recommendation engines increasingly rely on multimodal data to understand user preferences. Despite their potential, Multimodal Recommender Systems face three critical challenges. First, there is a significant overreliance on text; models often find textual similarity easier to compute, leading them to neglect visual nuances. Second, low image utilization stems from the fact that raw pixel data is less dense and harder for standard algorithms to interpret relationally compared to structured text. Finally, most systems rely on simplistic binary relationships (e.g., User A bought Item B), ignoring the richer "hyper-relations" formed when users interact with sets of items sharing common themes.
As illustrated in the figure above, items that appear similar in a vector space (based on text or raw image features) may not align with actual user needs in the real world. This gap necessitates a more sophisticated approach to feature representation and relational modeling.
The DOGE Framework
To address these limitations, DOGE introduces a two-pronged strategy focused on semantic enhancement and high-order relational modeling.
1. Semantic Relationship Enhanced Feature Generation
DOGE leverages the reasoning capabilities of Multimodal LLMs to act as a virtual knowledge base. By prompting an MLLM with both a product image and its title, the system generates descriptive characteristics and potential use cases. This output is transformed into a Semantic Modality, a high-density representation that aligns visual information with textual concepts. This modality serves as a bridge, effectively teaching the model the underlying connections between what an item looks like and how it is described.
2. Constructing the Hyper-Knowledge Graph (HKG)
Beyond standard graph structures, DOGE constructs a complex network incorporating five distinct relationship types: visual, textual, semantic, co-occurrence, and hyper-relational.
- Hyper-relations: The model treats the entire set of items a user interacts with as a "hyperedge." This captures collective item relationships that pair-wise links cannot represent.
- Dual Propagation: The architecture utilizes a LightGCN-based backbone for user-item interaction propagation, complemented by a specialized layer for homogeneous graphs (item-item and user-user) to refine embeddings.
Experimental Evaluation
Datasets
The model was rigorously tested on three Amazon datasets using a 5-core filtering setting:
- Baby: 19,445 users, 7,050 items.
- Home and Kitchen: 66,519 users, 28,237 items.
- Electronics: 192,403 users, 63,001 items.
Key Results
DOGE consistently outperformed 12 baseline models, including state-of-the-art systems like DRAGON and LGMRec. Across all metrics (Recall@K and NDCG@K), DOGE demonstrated an average improvement of 7.2% over the strongest baseline. Notably, the semantic modality proved exceptionally robust in the "Home and Kitchen" category, where metadata quality is typically lower, suggesting that LLM-enhanced features can effectively compensate for poor-quality text descriptions.
Discussion and Conclusion
The success of DOGE highlights the potential of LLMs as "feature enhancers" rather than simple text generators. By synthesizing image and text into a unified semantic layer, the model mitigates noise in raw visual data. Furthermore, the use of Hyper-Knowledge Graphs proves vital in sparse environments, such as the Electronics dataset (99.98% sparsity), where modeling multi-way relationships helps "fill in the blanks" left by sparse interaction data.
Future work will focus on addressing the computational overhead associated with MLLM feature generation and exploring dynamic hypergraphs that can evolve in real-time as user trends shift.
Summary updated