TranSUN: A Preemptive Paradigm to Eradicate Retransformation Bias Intrinsically from Regression Models in Recommender Systems

Jiahao Yu, Haozhuang Liu, Yeqiu Yang, Lu Chen, Wu Jian, Yuning Jiang, Bo Zheng

Abstract

In recommender systems, regression models often predict highly skewed targets like engagement duration or transaction amounts. To improve convergence, practitioners frequently apply non-linear transformations (e.g., log-transform) to the target. However, this introduces systematic retransformation bias due to Jensen's Inequality. We propose TranSUN, a preemptive framework that eliminates this bias during training via a jointly-learned auxiliary branch. TranSUN provides theoretical guarantees of unbiasedness and superior convergence. Extensive experiments on synthetic and large-scale industrial datasets, including Taobao, demonstrate that TranSUN effectively eradicates bias and achieves significant business gains, such as a +1.0% increase in Gross Merchandise Value (GMV).

Introduction

Regression models in large-scale recommender systems are tasked with predicting continuous variables like user dwell time or purchase amounts. These targets typically follow long-tailed distributions. To stabilize training, it is standard practice to transform the target $y$ using a function $T(\cdot)$, such as $T(y) = \log(y+1)$ or $\sqrt{y}$. While this aids convergence, it creates a "mathematical trap": the expectation of the transformed variable is not equal to the transformation of the expectation, i.e., $E[T(y)] \neq T(E[y])$. Consequently, applying the inverse transformation $T^{-1}$ to the model's output results in a biased estimate of the original target's mean. Traditional "post-hoc" (after-the-fact) corrections are often difficult to maintain and deploy in complex, high-traffic production environments. TranSUN moves this correction "inside" the model, fixing the bias before it ever leaves the training pipeline.

Methodology

TranSUN (Transformed-space Unbiased Network) addresses this by moving the correction into the model architecture. The framework introduces an auxiliary branch $z(x)$ alongside the main transformed regression model. This branch is trained to learn the ratio between the ground truth and the biased prediction.

A critical component of this architecture is the stop_grad operation. This operation decouples the two branches, ensuring that the bias-learning process does not interfere with the main model's ability to learn from the transformed distribution. Furthermore, the authors propose Generalized TranSUN (GTS), a theoretical framework that proves unbiasedness is maintained through "conditional linearity." This allows developers to plug in various loss functions (e.g., MAE, MSPE) while still guaranteeing an unbiased final output.

Figure 1: Illustration of biasedness and convergence of MSE, LogMSE, and TranSUN. TranSUN maintains a Predicted-to-Ground-Truth Ratio (PGR) near 1.0, indicating unbiasedness, while converging faster than standard MSE.

Experimental Results

The effectiveness of TranSUN was evaluated using a combination of synthetic distributions and massive real-world industrial data.

Synthetic and Benchmark Datasets

Testing across eight different synthetic distributions (right-skewed, left-skewed, and symmetric) confirmed the theoretical unbiasedness of the approach. The model was also validated on public benchmarks like CIKM16 (dwell time prediction) and DTMart (transaction amount prediction).

Industrial Impact

TranSUN was deployed on Taobao, serving over 300 million daily active users, to predict Gross Merchandise Value (GMV). In this context, 80% of the value typically comes from the top 30% of high-priced samples, making bias correction critical.

Key findings include:

  • Bias Eradication: On synthetic data, TranSUN kept the Signed Relative Error (SRE) below 0.7%. In contrast, standard transformed models (T-MSE) exhibited errors as high as 53% when using non-linear transformations.
  • Business Metrics: Deployment on Taobao resulted in a +1.0% GMV gain and a +0.9% increase in GMV Per Impression (GMVPI) compared to existing production baselines.
  • Efficiency: The method is lightweight, adding only approximately 2.7% to training time with negligible impact on inference latency.
Figure 5: Comparison results on synthetic data. Standard T-MSE fails significantly (noted in red) with non-linear transforms like log or square, while TranSUN remains stable and unbiased across all scenarios.

Discussion and Conclusion

TranSUN provides a robust, "preemptive" solution to retransformation bias. By integrating the correction intrinsically into the model training phase, it avoids the complexities of post-hoc calibration while delivering significant improvements in both predictive accuracy and online business metrics. While the current work focuses on right-skewed distributions common in e-commerce, future research will explore automated hyperparameter design and methods to mitigate slight trade-offs in ranking metrics like XAUC.

Summary updated

BibTeX
@inproceedings{yu2025transun,
  author = {Jiahao Yu and Haozhuang Liu and Yeqiu Yang and Lu Chen and Wu Jian and Yuning Jiang and Bo Zheng},
  title = {{TranSUN: A Preemptive Paradigm to Eradicate Retransformation Bias Intrinsically from Regression Models in Recommender Systems}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=3Swbyx5E7U},
}
Figure