PRISM: Personalized Recommendation via Information Synergy Module

Xinyi Zhang, Yutong Li, Peijie Sun, Letian Sha, Zhongxuan Han

TL;DR

PRISM is a "plug-and-play" module designed to supercharge multimodal recommendation systems by breaking down item information into four distinct categories: unique image features, unique text features, redundant (shared) info, and—most importantly—synergistic info that only emerges when modalities are combined. By dynamically weighting these components based on individual user preferences, PRISM achieves state-of-the-art accuracy and provides a rare window into why a specific item was recommended.

What Problem Does This Paper Address?

In the world of Sequential Recommendation (SR), models try to predict your next click based on your history. Multimodal models (MSR) try to improve this by looking at item images and descriptions. However, the authors identify two major gaps in current MSR approaches:

  1. The Synergy Gap: Most models focus on aligning modalities (finding what they share) or disentangling them (finding what makes them unique). They overlook "synergistic" information—the high-level properties that aren't visible in the image or text alone but appear when you look at both. For example, a "luxury gift" status might require seeing both a premium-looking bottle (image) and a prestigious vineyard origin (text).
  2. The "One-Size-Fits-All" Fusion: Existing models usually assume that the relative importance of an image versus text is the same for every user. In reality, one shopper might be driven by visual aesthetics (image uniqueness), while another is a spec-reader (text uniqueness).

Methods

The authors introduce PRISM (Personalized Recommendation via Information Synergy Module), which consists of two primary layers:

  • Interaction Expert Layer: This uses a Mixture-of-Experts (MoE) framework. Four dedicated "experts" are trained to extract specific types of information: Uni-I (Unique Image), Uni-T (Unique Text), Rdn (Redundant), and Syn (Synergy).
  • To force these experts to specialize, the authors use clever information-theoretic losses. For instance, the Synergy expert is trained to produce a prediction that is dissimilar to predictions made when one modality is masked, ensuring it only captures information that requires both inputs.
  • Adaptive Fusion Layer: Instead of just averaging the experts' outputs, this layer uses an interest-aware reweighting mechanism. It looks at the user’s historical behavior to decide which expert’s output should carry the most weight for that specific recommendation.

Datasets

The researchers evaluated PRISM on four widely-used public benchmarks:

  • Amazon Home, Beauty, and Sports: Captures purchasing behaviors and product reviews.
  • Yelp: Focuses on user feedback for restaurants and local services. All datasets were preprocessed using a "5-core" setting (ensuring users and items have at least five interactions) and evaluated using a standard leave-one-out protocol.

Key Insights and Ideas

  • Synergy is the "Secret Sauce": The paper demonstrates that synergistic information is not just "noise." It represents higher-level semantics that are crucial for accurate modeling. Ablation studies showed that removing the Synergy expert led to a significant drop in performance.
  • Information Decomposition (PID): The framework is grounded in Partial Information Decomposition theory. By treating multimodal fusion as a problem of decomposing information into unique, redundant, and synergistic parts, the model becomes much more robust than simple concatenation or attention-based fusion.
  • Interpretability Matters: Because the model assigns explicit weights to the four experts, we can actually see what drives a recommendation. The authors show a case study where a "collectible" item was correctly recommended because the model placed a high weight (40.27%) on the Synergy component.

Key Results

PRISM was tested as a plugin for three different "backbone" models (SASRec, STOSA, and InDiRec) and consistently outperformed all of them.

  • Performance Boost: On the Home dataset, PRISM improved the N@10 (Normalized Discounted Cumulative Gain) of the STOSA backbone by a staggering 71.43%.
  • Beating SOTA: PRISM+InDiRec outperformed recent state-of-the-art multimodal models like REARM and DGMRec across almost all metrics (Recall@10/20 and NDCG@10/20).
  • Visual Proof: Using t-SNE visualizations, the authors showed that PRISM successfully clusters item embeddings into four distinct groups, whereas previous methods failed to isolate synergistic information.

Limitations and Future Work

While PRISM is effective, the authors and the data suggest a few areas for growth:

  • Computational Overhead: Adding four experts and a reweighting module increases training time and GPU memory usage. While the authors argue this is "manageable" and scales linearly, it might be a hurdle for ultra-large-scale production systems.
  • Modality Expansion: The current study focuses on Image and Text. While the authors claim the framework is extensible, future work is needed to see how it handles more complex combinations like audio or video.
  • Temporal Dynamics: User preferences for different information types (e.g., caring more about text for technical goods but images for fashion) might change over time. The current model doesn't explicitly model the evolution of these preference weights.

Summary updated

BibTeX
@inproceedings{zhang2026prism,
  author = {Xinyi Zhang and Yutong Li and Peijie Sun and Letian Sha and Zhongxuan Han},
  title = {{PRISM: Personalized Recommendation via Information Synergy Module}},
  booktitle = {The Web Conference},
  year = {2026},
  doi = {10.1145/3774904.3792230},
}
Figure