Confidence-aware Fine-tuning of Sequential Recommendation Systems via Conformal Prediction

Chen Wang, Fangxin Wang, Ruocheng Guo, Yueqing Liang, Kay Liu, Philip S. Yu

Introduction

Sequential Recommendation Systems (SRecsys) are foundational to modern digital experiences, yet they often struggle with a critical aspect of user interaction: uncertainty. Traditional models are typically trained using Cross-Entropy (CE) loss, which focuses on maximizing the probability of a single ground-truth item. This approach leads to "confidence blindness," where the model cannot distinguish between high-certainty and low-certainty predictions. Furthermore, CE loss is misaligned with the practical "Top-K" nature of recommendations, treating the task as a winner-takes-all classification rather than a set-based retrieval problem.

The CPFT Framework

To address these issues, researchers have introduced CPFT (Confidence-aware Fine-Tuning). This framework integrates Conformal Prediction (CP)—a statistical method that provides mathematical coverage guarantees—directly into the model's training loop. Unlike post-hoc CP applications, CPFT uses differentiable proxy losses to optimize the model's internal representations for both accuracy and prediction efficiency.

The CPFT framework consists of a two-stage process: regular training followed by fine-tuning with conformal prediction losses (CPS and CPD) to refine the prediction set.

The methodology involves a two-stage process. First, a model (such as SASRec or BERT4Rec) undergoes standard training. In the second stage, it is fine-tuned using a combination of standard prediction loss and two novel CP-based losses:

  1. Conformal Prediction Set Size (CPS) Loss: This loss penalizes large, inefficient prediction sets, encouraging the model to be more decisive and produce smaller, high-confidence lists.
  2. Conformal Prediction Set Distance (CPD) Loss: This loss uses embedding similarity to pull the top-ranked items in the prediction set closer to the ground truth, refining the quality of the candidates.

Experimental Setup and Results

CPFT was evaluated on five real-world Amazon Review datasets (Scientific, Pantry, Instruments, Arts, and Office). The results demonstrate that CPFT significantly outperforms standard training across multiple metrics.

Training dynamics showing the convergence of prediction loss, CPS loss, CPD loss, and the stabilization of the coverage rate during the fine-tuning phase.

Key findings include:

  • Accuracy Gains: An average improvement of 4.55% in NDCG and Hit Rate, with particularly strong gains in Recall@50.
  • Efficiency: Fine-tuning with CPFT is highly efficient, requiring only 12.27% of the time needed for full training on the "Office" dataset.
  • Set Reduction: The framework successfully shrinks prediction sets (e.g., from ~300 to ~25 items) while maintaining the desired statistical coverage, effectively reducing the cognitive load on users.
  • Calibration: Qualitative analysis showed that CPFT-refined models assigned significantly higher confidence scores to relevant items (e.g., increasing from 0.0058 to 0.0466 for a ground-truth item).

Conclusion

CPFT represents a significant step toward more trustworthy and efficient recommendation systems. By leveraging Conformal Prediction as a regularizer during fine-tuning, the framework produces models that are not only more accurate but also better calibrated to their own uncertainty. Future work may explore non-exchangeable CP methods to better handle the temporal dependencies inherent in sequential data.

Summary updated

Abstract

In Sequential Recommendation Systems (SRecsys), traditional training approaches that rely on Cross-Entropy (CE) loss often prioritize accuracy but fail to align well with user satisfaction metrics. CE loss focuses on maximizing the confidence of the ground truth item, which is challenging to achieve universally across all users and sessions. It also overlooks the practical acceptability of ranking the ground truth item within the top-$K$ positions, a common metric in SRecsys. To address this limitation, we propose \textbf{CPFT}, a novel fine-tuning framework that integrates Conformal Prediction (CP)-based losses with CE loss to optimize accuracy alongside confidence that better aligns with widely used top-$K$ metrics. CPFT embeds CP principles into the training loop using differentiable proxy losses and computationally efficient calibration strategies, enabling the generation of high-confidence prediction sets. These sets focus on items with high relevance while maintaining robust coverage guarantees. Extensive experiments on five real-world datasets and four distinct sequential models demonstrate that CPFT improves precision metrics and confidence calibration. Our results highlight the importance of confidence-aware fine-tuning in delivering accurate, trustworthy recommendations that enhance user satisfaction.
BibTeX
@misc{wang2024confidence,
  author = {Chen Wang and Fangxin Wang and Ruocheng Guo and Yueqing Liang and Kay Liu and Philip S. Yu},
  title = {{Confidence-aware Fine-tuning of Sequential Recommendation Systems via Conformal Prediction}},
  year = {2024},
  eprint = {2402.08976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.08976},
}
Figure