NEMO-4-PAYPAL: Leveraging NVIDIA's Nemo Framework for empowering PayPal's Commerce Agent

Ali Sahami, Sudhanshu Garg, Andrew Wang, Chaitanya Kulkarni, Farhad Farahani, Sean Yun-Shiuan Chuang, Jian Wan, Srinivasan Manoharan, Uma Kona, Nitin Sharma, Linsey Pang, Prakhar Mehrotra, Jessica Clark, Mark MoyouarXiv 2025 · DOI · arXiv

Introduction

In the evolving landscape of digital retail, "agentic commerce" represents a shift from passive search bars to autonomous AI assistants capable of product discovery, catalog navigation, and transaction completion. PayPal, a leader in global payments, recently collaborated with NVIDIA to enhance its first-generation Commerce Agent. The primary objective was to overcome significant performance bottlenecks inherent in using general-purpose Large Language Models (LLMs) for specialized commerce tasks.

The Search and Discovery Bottleneck

PayPal's initial analysis revealed that the "Search and Discovery" component was responsible for over 50% of the total system response time. While traditional keyword-based search is insufficient for natural language intent, massive off-the-shelf LLMs introduced unacceptable latency—often exceeding 4.5 seconds—and high operational costs. To address this, the team sought a "Goldilocks" solution: a Small Language Model (SLM) that maintained high reasoning quality while operating within strict latency and cost constraints.

The Search and Discovery pipeline within the PayPal Commerce Agent, highlighting the role of the LLM in query understanding and attribute extraction.

Methodology: Fine-Tuning for Commerce

The core of the optimization strategy involved leveraging the NVIDIA NeMo Framework to specialize the llama3.1-nemotron-nano-8B-v1 architecture. The researchers employed a multi-stage optimization pipeline:

  1. Parameter-Efficient Fine-Tuning (PEFT): Using Low-Rank Adaptation (LoRA), the model was adapted to commerce-specific nuances without the prohibitive cost of full parameter retraining.
  2. Supervised Fine-Tuning (SFT): The model was trained on 10,000 high-quality prompt-response pairs synthesized from real customer chat interactions.
  3. Direct Preference Optimization (DPO): A reinforcement learning-style technique was applied using a preference-based dataset of 7,457 rows to refine the model's judgment between "better" and "worse" responses.
  4. Hypothetical Document Embeddings (HyDE): The SLM was trained to generate ideal product descriptions based on user queries, which were then used to query the actual product catalog, bridging the gap between natural language and structured data.

Agentic Orchestration and Infrastructure

Beyond model weights, the system's success relied on a robust orchestration layer. The Agent Orchestrator manages short-term and long-term memory, allowing the assistant to maintain context across a shopping session. This is integrated with a personalization engine and a product catalog to ensure recommendations are both relevant and accurate.

The Agent Orchestrator architecture, integrating short-term and long-term memory with a specialized LLM core.

To maximize throughput, the system was benchmarked on NVIDIA H100 and B200 (Blackwell) GPUs. Utilizing 3D parallelism (Tensor, Pipeline, and Data parallelism) through NVIDIA NIM, the team achieved a 35% latency improvement from hardware upgrades alone, complementing the software-level optimizations.

Key Results and Impact

The transition to the fine-tuned Nemotron SLM yielded dramatic improvements in production metrics:

  • Latency Reduction: Overall agent latency dropped by 49%, with retrieval-specific latency improving by 58%. This brought the system to a sub-2-second response target.
  • Cost Efficiency: Monthly GPU operational costs were reduced by 45% compared to the previous LLM-based architecture.
  • Quality Gains: Despite the smaller parameter count, the SFT-tuned model achieved a 23% improvement in quality score over the non-fine-tuned 8B baseline, as measured by "LLM-as-a-Judge" metrics.

Conclusion and Future Work

This collaboration demonstrates that domain-specific specialization can allow smaller models to outperform larger, general-purpose counterparts in production environments. By focusing on the query formulation stage—the primary bottleneck in retrieval-heavy systems—PayPal successfully balanced performance, cost, and quality. Future efforts will focus on extending these techniques to other specialized agents within the PayPal ecosystem, including post-purchase support and personalized ranking systems.

Summary updated

Abstract

We present the development and optimization of PayPal's Commerce Agent, powered by NEMO-4-PAYPAL, a multi-agent system designed to revolutionize agentic commerce on the PayPal platform. Through our strategic partnership with NVIDIA, we leveraged the NeMo Framework for LLM model fine-tuning to enhance agent performance. Specifically, we optimized the Search and Discovery agent by replacing our base model with a fine-tuned Nemotron small language model (SLM). We conducted comprehensive experiments using the llama3.1-nemotron-nano-8B-v1 architecture, training LoRA-based models through systematic hyperparameter sweeps across learning rates, optimizers (Adam, AdamW), cosine annealing schedules, and LoRA ranks. Our contributions include: (1) the first application of NVIDIA's NeMo Framework to commerce-specific agent optimization, (2) LLM powered fine-tuning strategy for retrieval-focused commerce tasks, (3) demonstration of significant improvements in latency and cost while maintaining agent quality, and (4) a scalable framework for multi-agent system optimization in production e-commerce environments. Our results demonstrate that the fine-tuned Nemotron SLM effectively resolves the key performance issue in the retrieval component, which represents over 50\% of total agent response time, while maintaining or enhancing overall system performance.
BibTeX
@misc{sahami2025nemo,
  author = {Ali Sahami and Sudhanshu Garg and Andrew Wang and Chaitanya Kulkarni and Farhad Farahani and Sean Yun-Shiuan Chuang and Jian Wan and Srinivasan Manoharan and Uma Kona and Nitin Sharma and Linsey Pang and Prakhar Mehrotra and Jessica Clark and Mark Moyou},
  title = {{NEMO-4-PAYPAL: Leveraging NVIDIA's Nemo Framework for empowering PayPal's Commerce Agent}},
  year = {2025},
  eprint = {2512.21578},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.21578},
}
Figure