TL;DR
This paper introduces EPM-RL, a novel reinforcement learning framework for on-premise product mapping in e-commerce. It distills complex, expensive LLM-based reasoning into a more efficient, deployable in-house model, achieving a better quality-cost trade-off than existing API-based solutions.
What Problem Does This Paper Address?
In e-commerce, accurately matching product listings to a single underlying product is crucial for tasks like price monitoring and inventory management. However, this "product mapping" is incredibly challenging due to how sellers title their products. They often stuff titles with promotional keywords, platform-specific jargon, and bundle details, making identical products appear under vastly different names. Conversely, subtle differences in variants (like size or color) can make dissimilar products seem alike. While recent LLM-based and multi-agent approaches show promise, they often rely on costly external APIs, complex inference-time orchestration, and raise privacy concerns, making them unsuitable for many enterprise settings. This paper aims to bridge this gap by creating an accurate, efficient, and privacy-preserving on-premise solution.
Methods
The core innovation of EPM-RL lies in its two-stage training process:
- Parameter-Efficient Fine-Tuning (PEFT): The authors start with human-labeled product pairs and LLM-generated rationales. They then use PEFT (specifically LoRA) to fine-tune a smaller "student" model. This stage focuses on teaching the model to generate structured reasoning traces (e.g.,
<reason>...</reason>) alongside the final product match label (<label>...</label>). This structured output is key for interpretability and further optimization. - Reinforcement Learning (RL) with GRPO: Building on the PEFT stage, the model is further refined using Group Relative Policy Optimization (GRPO). Here, the model generates multiple reasoning outputs ("rollouts") for a given input, and these are scored by a reward function. This reward is a weighted combination of:
- Format Compliance: Does the output adhere to the
<reason>and<label>structure? - Label Correctness: Is the predicted match label accurate?
- LLM-as-a-Judge: This is a novel component where three specialized LLM "judge agents" evaluate the quality of the reasoning itself. These judges focus on specific aspects: core product identity, model/identifier matching, and variant conflict detection. This "judge-based" reward encourages the model to produce not just correct answers, but also faithful and informative reasoning.
This approach distills the complex reasoning of larger models into a trainable, on-premise system.
Datasets
The paper uses an internally collected and curated product mapping benchmark consisting of approximately 12,000 labeled product pairs. This dataset is described as diverse, covering around 500 brands and reflecting real-world variations in naming conventions, promotional language, and bundle descriptions. The dataset is partitioned into splits for PEFT training (6K), RL training (4K), validation (1K), and testing (1K).
Key Insights and Ideas
- Distilling Agentic Reasoning: The central idea is that complex, multi-step reasoning performed by expensive LLM agents can be "distilled" into a more efficient, single model through PEFT and RL.
- Structured Reasoning is Key: Forcing the model to output explicit reasoning traces, not just a binary label, is crucial for both interpretability and for enabling effective reward shaping in RL.
- LLM-as-a-Judge for Reasoning Quality: Using specialized LLMs to score the quality of the generated reasoning (beyond just correctness) is a powerful way to guide the model towards more faithful and useful explanations.
- On-Premise Viability: The framework is designed for on-premise deployment, addressing privacy and cost concerns associated with relying on external APIs.
- Quality-Cost Trade-off: EPM-RL aims to offer a better balance between performance and operational cost compared to existing solutions.
Key Results
The paper reports strong results on their internal benchmark:
- EPM-RL (specifically the LORA + GRPO Reasoning variant) achieves the best overall F1 score (0.8120) among all tested methods.
- It significantly outperforms PEFT-only training, demonstrating the benefit of the RL refinement stage.
- EPM-RL offers a better quality-cost trade-off than commercial API-based baselines.
- While some baselines (like Multi-Agent RAG and GPT-5.4 reasoning) achieve very high precision, they often suffer from lower recall. EPM-RL strikes a better balance, achieving competitive precision with substantially higher recall.
Limitations and Future Work
The authors acknowledge several limitations:
- Internal Benchmark Scope: The evaluation is primarily on an internal dataset. Further testing on public, cross-domain benchmarks is needed to assess generalization.
- Limited Qualitative Analysis: The paper could benefit from more in-depth case studies to understand failure modes, reward sensitivity, and the faithfulness of the generated reasoning.
- Incomplete System and Ablation Coverage: A full ablation study of reward components, judge prompts, and training hyperparameters is missing. A detailed analysis of latency/cost trade-offs across different deployment settings is also suggested for future work.
Summary updated