Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

Jujia Zhao, Zihan Wang, Shuaiqun Pan, Suzan Verberne, Zhaochun Ren

Abstract

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distorts general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces Multi-Subspace Decomposition, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, and Null-Space Projection, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks.

Introduction: The Challenge of Unified S&R

In modern online ecosystems, Search and Recommendation are two sides of the same coin. Search addresses explicit user intent via queries, while Recommendation infers implicit intent from historical behaviors. Unifying these into a single LLM-based framework promises a more holistic understanding of user needs. However, standard parameter-efficient fine-tuning (PEFT) methods like LoRA encounter significant obstacles.

First, Gradient Conflict arises because search and recommendation often have divergent goals. Search focuses on immediate query-item relevance, whereas recommendation emphasizes long-term user preferences. This leads to a "tug-of-war" in the optimization space, where gradients for one task may negate progress in the other. Second, Intent Shift (Knowledge Erosion) occurs when fine-tuning on specific S&R data causes the model to overfit, distorting the general-domain reasoning capabilities that make LLMs powerful. A model might become a proficient recommender but lose its ability to parse complex, nuanced search queries it could previously handle.

Gradient conflict analysis and examples of intent shift in standard PEFT methods.

Methodology: Gradient Multi-Subspace Tuning (GEMS)

To address these issues, we introduce GEMS, a framework that avoids full fine-tuning in favor of a surgical update strategy. GEMS operates on three primary components:

Multi-Subspace Decomposition

GEMS disentangles optimization signals into three distinct low-rank subspaces using Singular Value Decomposition (SVD):

  • Shared Subspace: Captures patterns beneficial to both tasks, such as general item categories.
  • Search-specific Subspace: Focuses on query-item alignment.
  • Recommendation-specific Subspace: Focuses on long-term user interaction history. An Adaptive Gating Mechanism balances these updates dynamically based on the training loss of each task, effectively resolving the gradient "tug-of-war."

Null-Space Projection

To preserve the model's core reasoning abilities, GEMS identifies the "principal representation space" of the pre-trained LLM. It then constrains fine-tuning updates to the null space—directions orthogonal to the general-domain knowledge space. This ensures that the model's fundamental intelligence remains intact while it learns new S&R tasks.

The GEMS framework architecture featuring Multi-Subspace Decomposition and Null-Space Projection.

Experimental Evaluation

We evaluated GEMS on two benchmark datasets: Amazon (Electronics), a standard product review dataset with synthesized search queries, and Qilin, a real-world social media dataset containing actual user queries and interaction histories.

Key Results

  • Superior Performance: GEMS consistently outperformed state-of-the-art baselines like GenSAR and BSR. On the T5-base backbone, GEMS outperformed the best specialized baselines by over 470% on Hit@5 for certain tasks.
  • Conflict Mitigation: Empirical analysis showed that GEMS reduced the average gradient conflict magnitude by over 85% compared to standard PEFT methods.
  • Reasoning Stability: Consistency analysis revealed that GEMS reduced the disruption of previously correct answers by 10-14% compared to LoRA, proving its ability to maintain the LLM's original "intelligence" while adapting to new domains.

Conclusion and Future Work

GEMS demonstrates that it is possible to unify search and recommendation in LLMs efficiently without sacrificing general reasoning or suffering from task interference. Future research will explore selective preservation of domain-relevant knowledge and dynamic adjustment of subspace ranks to further enhance adaptation efficiency. While currently validated on models up to 3 billion parameters (Qwen-3B), scaling GEMS to larger frontier models remains a key objective.

Summary updated

Abstract

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.
BibTeX
@inproceedings{zhao2026unifying,
  author = {Jujia Zhao and Zihan Wang and Shuaiqun Pan and Suzan Verberne and Zhaochun Ren},
  title = {{Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2026},
  doi = {10.1145/3805712.3809719},
}
Figure