Hierarchical User Intent Graph Network forMultimedia Recommendation

Yinwei Wei, Xiang Wang, Xiangnan He, Liqiang Nie, Yong Rui, Tat-Seng Chua

Abstract

Modern multimedia recommendation systems often struggle to bridge the gap between raw features (visual, acoustic, textual) and actual user motivations. This paper introduces the Hierarchical User Intent Graph Network (HUIGN), a framework designed to explicitly model multi-level user intents. By organizing these intents into a hierarchy—ranging from fine-grained attributes like specific actors to coarse-grained concepts like genre—HUIGN captures the complex structure of personal taste. Experimental results on Movielens, TikTok, and Kwai datasets demonstrate that HUIGN significantly outperforms state-of-the-art baselines, providing more accurate and semantically meaningful recommendations.

Introduction

Most contemporary multimedia recommenders utilize "off-the-shelf" features from pre-trained models like ResNet or BERT. While powerful for general tasks, these features are not optimized for understanding the nuanced "why" behind a user's interaction. User intent is inherently hierarchical; a user might be drawn to a video because of a specific visual quality (fine-grained) or a broader storyline (coarse-grained). Existing models often treat these features as a flat vector, failing to capture the multi-level nature of human preference. HUIGN addresses this by learning disentangled, hierarchical representations of user intent directly from interaction data and item content.

Methodology: The HUIGN Framework

The HUIGN architecture operates through a series of graph-based aggregations to distill intent from multi-modal signals.

The HUIGN architecture illustrating intra-level and inter-level aggregation across visual, acoustic, and textual modalities.

The process involves three key stages:

  1. Co-interacted Item Graph Construction: Items are connected in a graph if they share common users, embedding behavioral patterns into the item representations.
  2. Recursive Aggregation:
  3. Intra-level Aggregation: A light Graph Convolutional Network (GCN) variant aggregates signals from neighboring nodes to capture local "vibes" or similar item characteristics.
  4. Inter-level Aggregation: Higher-level "supernodes" are formed to summarize lower-level intents into broader categories, creating a hierarchical structure.
  5. Disentangled Learning: To ensure intents are distinct, the model employs a cross-entropy loss to sharpen item-intent associations and an independence loss to minimize redundancy between different learned intents.

Experimental Evaluation

The model was tested on three diverse datasets: Movielens (enriched with multi-modal metadata), TikTok, and Kwai. These platforms represent a mix of traditional movie recommendations and modern micro-video services.

Key Results

HUIGN consistently achieved superior performance compared to competitive baselines such as MMGCN and DisenGCN.

Performance comparison of HUIGN against state-of-the-art baselines across three datasets.

As shown in the results table, HUIGN provides substantial relative improvements in NDCG@10: 9.35% on Movielens, 12.10% on TikTok, and 10.70% on Kwai. These gains highlight the effectiveness of modeling intent as a hierarchy rather than a flat feature set.

Discussion and Insights

  • Hierarchical Superiority: Ablation studies confirmed that a multi-layer hierarchy (e.g., three layers consisting of 32, 8, and 4 intents respectively) outperforms single-layer models, mirroring human cognitive categorization.
  • Modality Impact: In micro-video contexts, visual features were found to be the most influential modality, though the integration of acoustic and textual data further refined accuracy.
  • Robustness through Disentanglement: The independence loss was critical for maintaining performance across varying graph densities, ensuring that the learned intents remained distinct and informative.

Conclusion

The Hierarchical User Intent Graph Network (HUIGN) represents a significant step forward in multimedia recommendation by moving beyond raw feature filtering to intent-based representation. By explicitly modeling the hierarchical nature of user motivation, the system provides more robust and interpretable recommendations. Future work will explore more sophisticated modality fusion techniques and the integration of external knowledge graphs to further bridge the semantic gap.

Summary updated

Abstract

Understanding user preference on item context is the key to acquire a high-quality multimedia recommendation. Typically, the pre-existing features of items are derived from pre-trained models (e.g. visual features of micro-videos extracted from some neural networks), and then introduced into the recommendation framework (e.g. collaborative filtering) to capture user preference. However, we argue that such a paradigm is insufficient to output satisfactory user representations, which hardly profile personal interests well. The key reason is that present works largely leave user intents untouched, then failing to encode such informative representation of users. In this work, we aim to learn multi-level user intents from the co-interacted patterns of items, so as to obtain high-quality representations of users and items and further enhance the recommendation performance. Towards this end, we develop a novel framework, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Hierarchical User Intent Graph Network</i> , which exhibits user intents in a hierarchical graph structure, from the fine-grained to coarse-grained intents. In particular, we get the multi-level user intents by recursively performing two operations: 1) intra-level aggregation, which distills the signal pertinent to user intents from co-interacted item graphs; and 2) inter-level aggregation, which constitutes the supernode in higher levels to model coarser-grained user intents via gathering the nodes’ representations in the lower ones. Then, we refine the user and item representations as a distribution over the discovered intents, instead of simple pre-existing features. To demonstrate the effectiveness of our model, we conducted extensive experiments on three public datasets. Our model achieves significant improvements over the state-of-the-art methods, including MMGCN and DisenGCN. Furthermore, by visualizing the item representations, we provide the semantics of user intents.
BibTeX
@article{wei2022hierarchical,
  author = {Yinwei Wei and Xiang Wang and Xiangnan He and Liqiang Nie and Yong Rui and Tat-Seng Chua},
  title = {{Hierarchical User Intent Graph Network forMultimedia Recommendation}},
  journal = {IEEE Trans. Multimedia},
  year = {2022},
  url = {https://openreview.net/forum?id=xXKHSRE4K6v},
}
Figure