BeFA: A General Behavior-driven Feature Adapter for Multimedia Recommendation

Qile Fan, Penghang Yu, Zhiyi Tan, Bing-Kun Bao, Guanming Lu

Abstract

Multimedia recommender systems (MRS) rely on content features extracted by pre-trained encoders to supplement behavioral data. However, these general-purpose encoders often capture irrelevant background details, leading to "information drift." We introduce the Behavior-driven Feature Adapter (BeFA), a plug-and-play module that utilizes user behavior as a supervisory signal to filter and refine content features. Experimental results across three real-world datasets demonstrate that BeFA significantly improves recommendation accuracy with minimal computational overhead.

1. Introduction: The Information Drift Problem

Standard multimedia recommendation pipelines utilize pre-trained models like ResNet or CLIP to extract features from item images. While powerful, these encoders are optimized for general classification tasks rather than recommendation. This leads to two primary challenges: Information Drift, where the model focuses on irrelevant background elements, and Information Omission, where subtle but critical product details are ignored.

Illustration of Information Drift: The encoder prioritizes background elements (Park) over the user-preferred product (Cheongsam).

As shown in the figure above, an encoder might assign a high relevance score to a "Park" background (0.94) while failing to capture the "Cheongsam" design (0.11) that actually attracts the user. When these noisy features are fused with behavioral data, they degrade the performance of the recommender system.

2. Methodology: Behavior-driven Adaptation

To address these issues, the authors propose BeFA, which operates on a "decouple-filter-reconstruct" principle.

2.1 Similarity-Based Attribution Analysis

Before adaptation, the authors utilize a novel attribution tool to visualize the gap between encoder focus and user behavior. By calculating the cosine similarity between pixel-level features and behavioral embeddings, they generate heatmaps that identify which image regions truly align with user interactions.

2.2 The BeFA Pipeline

BeFA refines features through three main stages:

  1. Decoupling: Content features are projected into a decoupled space to isolate distinct semantic components.
  2. Filtering: A gating mechanism, guided by user and item ID embeddings (behavioral signals), weights these components to emphasize preference-relevant features.
  3. Reconstructing: The filtered components are selectively recombined into a purified representation for downstream tasks.

3. Experimental Evaluation

BeFA was evaluated on the TMALL, Microlens, and H&M datasets using CLIP and ImageBind as base encoders.

3.1 Key Results

The integration of BeFA led to substantial performance gains across various base models (e.g., BM3, MGCN). On average, the module provided a 9.07% improvement in Recall@20 and an 11.02% improvement in NDCG@20. Notably, BeFA is highly efficient, increasing tunable parameters by only ~1-2% and training time by approximately 15%.

3.2 Comparison with LoRA

Unlike standard fine-tuning methods like Low-Rank Adaptation (LoRA), BeFA explicitly addresses the "focus" problem. While LoRA adapts the weights of the encoder, BeFA uses behavioral guidance to actively filter the output features, proving more effective for recommendation tasks.

4. Conclusion and Future Work

BeFA demonstrates that user behavior is the ultimate ground truth for content relevance in recommendation. By "purifying" features through a behavioral lens, BeFA bridges the gap between general-purpose vision models and specific user preferences. Future research will explore learnable decoupling spaces and extending this behavior-driven filtering to text and audio modalities.

Summary updated

BibTeX
@inproceedings{fan2025befa,
  author = {Qile Fan and Penghang Yu and Zhiyi Tan and Bing-Kun Bao and Guanming Lu},
  title = {{BeFA: A General Behavior-driven Feature Adapter for Multimedia Recommendation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i11.33266},
}
Figure