TL;DR
DIGCN (Dual-granularity Intermediate fusion GCN) improves multimodal recommendation by using Large Language Models (LLMs) to generate coarse-grained raw features before training, bypassing the difficulty of learning complex representations from sparse interaction data. It introduces an intermediate fusion mechanism that dynamically weights different modalities during graph propagation, preserving unique modality-specific information while capturing cross-modal signals.
What Problem Does This Paper Address?
Multimodal recommender systems aim to understand user preferences by combining collaborative filtering (user-item interactions) with content features like product images and descriptions. However, they face two major hurdles:
- The Representation Bottleneck: Learning "multi-granularity" representations (e.g., distinguishing between a specific "floral summer dress" and the general category of "clothing") is difficult because user-item interaction data is notoriously sparse. There isn't enough supervisory signal to train these complex features from scratch.
- The Fusion Bottleneck: Existing models usually fuse modalities either too early (before the Graph Convolutional Network) or too late (after the GCN). Furthermore, many rely on contrastive learning to "align" modalities, which can accidentally over-compress data and erase the unique, discriminative features that make a specific modality useful for recommendation.
The authors suggest we "Take a Step Back"—instead of trying to learn these complexities during training, we should leverage the zero-shot reasoning of LLMs to prepare the data better and use a more flexible fusion architecture.
Methods
The proposed DIGCN framework consists of three core components:
- Dual-Granularity Raw Feature Construction: Instead of asking the model to learn granularities, the authors use LLMs (GPT-4o for text) and MLLMs (DALL-E-3 for images) to generate "coarse-grained" versions of the raw data. For example, a verbose product description is abstracted into its core semantics. This results in two sets of raw features: the original fine-grained features and the LLM-generated coarse-grained features.
- Intermediate Fusion GCN: Unlike standard models that fuse once, DIGCN performs fusion at every layer of the GCN. It treats the user/item ID (behavioral modality) as an anchor. At each layer, it calculates a dynamic cross-modality weight correction based on the cosine similarity between the anchor and the content modalities (text/visual). This allows the model to adaptively decide which modality is most important at different depths of the graph.
- Item-Item Semantic Graph: To further combat data sparsity, the model pre-computes a frozen similarity graph between items based on their raw features. This provides a "shortcut" for information to flow between similar items, even if they don't share many common users.
Datasets
The model was evaluated on four widely used Amazon Review datasets, filtered using the standard "5-core" setting (users and items must have at least 5 interactions):
- Baby
- Sports and Outdoors
- Pet Supplies
- Office Products
These datasets vary in scale and sparsity (all >99.5% sparse), providing a robust test for recommendation performance.
Key Insights and Ideas
- LLMs as Feature Engineers: The "aha moment" here is realizing that LLMs are better at abstracting semantics than a recommendation model trained on sparse clicks. By moving abstraction to a pre-processing step, the model can focus on the recommendation task itself.
- The Anchor Strategy: By using the ID embedding as an anchor for fusion, the model avoids the "alignment trap." Instead of forcing text and images to look like each other, it forces them to support the behavioral signal, preserving their unique strengths.
- Efficiency via Freezing: By pre-constructing the item-item graph and the dual-granularity features offline, DIGCN achieves better performance than state-of-the-art models while maintaining competitive training times and lower memory usage.
Key Results
- Superiority over Baselines: DIGCN consistently outperformed 15 strong baselines (including MambaRec and LGMRec). It showed a relative improvement in Recall@20 of up to 8.81% on the Pet dataset.
- Cold-Start Robustness: In scenarios where 20% of items were completely hidden during training (item cold-start), DIGCN maintained a significant lead, proving that the LLM-generated coarse features provide a strong semantic safety net.
- Sparsity Resilience: The model showed the largest gains for users with the fewest interactions (0-5 training samples), validating the effectiveness of the dual-granularity approach in low-data regimes.
- Extensibility: While DIGCN doesn't require contrastive learning, the authors showed that adding optional alignment tasks (like InfoNCE) can further boost performance, making it a flexible foundation for future research.
Limitations and Future Work
- LLM Stability: The authors noted that while adding more granularities (3 or 4 levels) could theoretically help, LLMs become less stable and more prone to randomness at higher abstraction levels, leading to diminishing returns.
- Computational Cost of Pre-processing: While training is efficient, the one-time cost of calling LLM APIs for every item in a massive catalog (millions of items) could be significant for production systems.
- Future Directions: The authors suggest exploring more sophisticated ways to handle the inherent randomness of LLM outputs and investigating how to better integrate the "anchor" mechanism with other graph architectures beyond GCNs.
Summary updated