Abstract
Recent advances in Large Language Models (LLMs) have opened new avenues for sequential recommendation by enabling natural language reasoning over user behavior sequences. A common approach formulates recommendation as a language modeling task, where interaction histories are transformed into prompts and user preferences are learned via supervised fine-tuning. However, these methods operate solely in the textual modality and often miss users' fine-grained interests, especially when shaped by rich visual signals such as product images or movie posters. Multimodal Large Language Models (MLLMs) offer a promising alternative by aligning text and vision in a shared semantic space. A prevalent training paradigm applies Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO) to model user preferences. Yet, two core challenges remain: 1) Imbalanced sample hardness, where random negative sampling causes overfitting on easy examples and under-training on hard ones; 2) Cross-modal semantic bias, where the fixed reference model in DPO prevents the policy model from correcting modality misalignments—especially over long sequences. To address these issues, we propose a Multimodal LLM framework that integrates Hardness-aware and Noise-regularized preference optimization for Recommendation (HaNoRec). Specifically, HaNoRec dynamically adjusts optimization weights based on both the estimated hardness of each training sample and the policy model's real-time responsiveness, prioritizing harder examples. It further introduces Gaussian-perturbed distribution optimization on output logits to enhance cross-modal semantic consistency and reduce modality bias inherited from the reference model.
Introduction
Sequential Recommendation (SR) aims to predict a user's next interaction based on their historical behavior. While traditional LLMs excel at reasoning over text-based histories, they frequently overlook the visual cues—such as movie posters or product aesthetics—that significantly influence human preferences. Multimodal Large Language Models (MLLMs) attempt to bridge this gap by integrating visual encoders with language backbones to process both textual metadata and visual features.
Despite their potential, MLLMs face significant hurdles during the Direct Preference Optimization (DPO) phase. Standard training often relies on random negative sampling, leading to an "imbalanced sample hardness" problem where the model overfits to easy distinctions and fails to learn subtle user preferences. Furthermore, a "cross-modal semantic bias" often emerges, where the alignment between text (titles) and vision (images) degrades over long sequences, a phenomenon the authors term the "weak get weaker" trend.
Methodology: The HaNoRec Framework
To overcome these limitations, the researchers introduced HaNoRec, a framework featuring two primary innovations designed to refine the DPO pipeline:
Hardness-aware Reweighting Strategy (HaRS)
HaRS moves beyond uniform treatment of training pairs. It dynamically calculates a scaling factor based on two metrics: Data Hardness, estimated by the overlap between positive and negative item clusters in the semantic space, and Model Responsiveness, a real-time measure of how much the model is learning from a specific sample. By prioritizing "hard" examples—items that are similar to the user's history but were not chosen—HaRS ensures the model focuses on informative, high-signal data.
Noise-regularized Distribution Optimization (NoDO)
To mitigate modality bias, NoDO injects Gaussian noise into the trainable LoRA (Low-Rank Adaptation) weights during training. This noise acts as a regularizer, smoothing output distributions. When paired with an adaptive KL-divergence constraint, it encourages the model to align title and image embeddings toward a shared semantic center, preventing the representation drift common in standard DPO where poorly aligned items often see their quality degrade further.
Experimental Evaluation
The framework was evaluated across three diverse datasets: Microlens (short videos), Netflix (movies), and Movielens-1M (re-crawled for high-quality posters). As illustrated in Figure 1, MLLMs consistently outperform text-only LLMs and soft-prompting methods in terms of Hit Ratio across various domains. HaNoRec further amplifies these gains, demonstrating superior performance across all metrics, including AUC, HR@3, and NDCG@3.
Key findings include:
- Significant Performance Gains: On the Microlens dataset, HaNoRec achieved an 11.02% improvement in NDCG@3 over existing state-of-the-art MLLM methods.
- Qualitative Coherence: Blind tests using GPT-4o revealed that HaNoRec's recommendations were more thematically and stylistically aligned with user history compared to standard models, which often defaulted to generic popular items.
- Model Agnostic Scaling: The optimization strategy proved effective across different model sizes, providing a consistent boost to both 3B and 7B parameter backbones, such as Qwen-2.5-VL and LLaVA-NeXT.
Conclusion
HaNoRec addresses critical flaws in the preference optimization of MLLMs for recommendation. By dynamically managing sample hardness and regularizing cross-modal alignment, it significantly improves the accuracy and relevance of sequential recommendations. While current constraints limit user history length due to computational intensity, future work will focus on extending this framework to handle longer contexts and more complex video-level multimodal signals.
Summary updated