Curvature as Self-Normalization for Debiased Recommender Training

Recommender systems learn from ratings that users chose to give. People rate what they already like or were shown, so the observed data is "missing not at random." The standard fix reweights each observed rating by the inverse of the estimated probability that it was observed, called its propensity. A rating of a niche item that almost nobody sees can receive a weight in the hundreds. These weights make the training objective unbiased, but they also make gradients explode on exactly the parameters that belong to rare items. The field has responded with weight clipping, stabilization constraints, and calibration layers, and each of these trades some of the unbiasedness back for stability.

This note proposes a different place to absorb the weights: the optimizer's estimate of curvature, meaning how sharply the loss bends along each parameter. For a propensity-weighted loss, the diagonal of the Gauss-Newton matrix (a positive approximation to the Hessian, the matrix of second derivatives) is itself an inverse-propensity-weighted sum. Dividing the weighted gradient by this weighted curvature, coordinate by coordinate, gives an update that is a ratio of two weighted sums. Statisticians call this kind of ratio a self-normalized, or Hájek, estimator, and it is far less sensitive to extreme weights than the raw weighted sum. Preconditioning by a positive diagonal does not move the stationary points of the objective, so the debiased target is left intact. The claim is that diagonal curvature preconditioning gives the variance control people currently buy with clipping, without the bias that clipping adds. The note also explains why Adam's squared-gradient normalization does not deliver the same property.

Core Claim

Let $w_{ui} = 1/\hat P_{ui}$ be the inverse estimated propensity for user $u$ and item $i$. The inverse propensity scoring (IPS) objective, as formalized for recommendation by Schnabel et al., sums $w_{ui}\,\ell(y_{ui}, f_\theta(u,i))$ over observed pairs, where $\ell$ is the per-rating loss and $f_\theta$ the model's prediction. Its diagonal Gauss-Newton curvature for parameter $\theta_j$ is

$$h_j = \lambda + \sum_{(u,i)\ \text{observed}} w_{ui}\, \ell''_{ui}\, \Big(\tfrac{\partial f_\theta(u,i)}{\partial \theta_j}\Big)^2 ,$$

where $\lambda$ is the L2 regularization strength and $\ell''_{ui}$ is the second derivative of the loss with respect to the prediction. The doubly robust (DR) objective has the same weighted structure on its residual term. A preconditioned step $\Delta\theta_j = -\eta\, m_j / \max(h_j,\gamma)$ uses $m_j$, the moving average of the weighted gradient; $\eta$, a step size; and $\gamma$, a floor as in Sophia.

Consider an item embedding under squared loss. This step is a weighted least-squares update: weighted residuals divided by weighted information. If every observation of that item is multiplied by a common large weight, the step does not change at all, while the fixed point still reflects the relative weights across observations.

Adam behaves differently. Its normalizer is the square root of a moving average of $w^2 g^2$, where $g$ is the gradient. So it scales with the second moment of the weights rather than their first. A single high-weight rating inflates that average for thousands of steps, given Adam's usual decay rate, and suppresses learning from that item's other ratings.

The proposal therefore makes three predictions. First, curvature normalization reduces the variance of the learned model under extreme propensities. Second, it keeps the bias at the level of the unclipped objective. Third, its advantage over Adam grows with the spread of the weights.

Relation to Prior Work

The motivating evidence comes from the debiasing literature. Doubly Robust Joint Learning combines an error-imputation model with IPS-weighted residuals and identifies small propensities as the main source of variance. StableDR bounds that variance by constraining the propensity model inside a three-model update cycle, and argues that simple clipping buys stability at the cost of bias. Relaxing the Accurate Imputation Assumption adds per-user and per-item balancing constraints. Two calibration papers, the Doubly Calibrated Estimator and Cali-MR, calibrate propensity and imputation models with expert layers and bi-level optimization. All of these intervene on the estimator or on the propensities. None of them changes how the prediction model's parameters move given the weights. Our proposal is orthogonal: it can sit underneath any of these estimators.

The method itself comes from the curvature literature. Sophia supplies the clipped diagonal-Newton update and the Gauss-Newton-Bartlett estimator. HesScale supplies an exact output-layer diagonal propagated backward in linear time, and shows that Adam's squared gradient is a poor proxy for the true diagonal. Both were developed for unweighted losses, and neither asks what happens when per-sample weights span orders of magnitude.

AGD is the only curvature optimizer in the file that was tested on recommendation data, specifically the Avazu and Criteo click-through datasets. It treated those as ordinary supervised problems and never applied propensity weights.

The closest conceptual relative is Boosting Prediction with Data Missing Not at Random, which combines inverse-propensity weighting with boosting. Second-order boosting computes leaf values as ratios of summed gradients to summed Hessians, which is the same ratio structure we exploit. That paper stays with low-dimensional tree models and does not analyze this ratio's variance behavior. We bring the idea to latent-factor and neural recommenders, and we test it explicitly.

We set aside the bi-level route. That route would use diagonal Hessians to approximate Cali-MR's inverse-Hessian hypergradient, in the spirit of the matrix-free approach in An Augmented Lagrangian Approach to Bi-Level Optimization. It would mainly be a speedup with a known recipe, whereas the self-normalization question bears on what debiased training actually learns.

Research Program

The first component is analytic. For matrix factorization with squared and logistic loss, we will derive the per-coordinate step under IPS and DR weighting for three normalizers: curvature, Adam, and none (plain SGD). We will characterize each step's sensitivity to a single extreme weight, measured by how much and for how long one high-weight rating shifts an item embedding. We will also derive the per-coordinate effective sample size, $(\sum w h)^2 / \sum w^2 h^2$, where $h$ is the per-rating curvature contribution. This is the quantity that should predict where curvature normalization helps most.

The second component is the optimizer. For matrix factorization, the Gauss-Newton diagonal is closed-form and essentially free to compute. For neural collaborative filtering (NCF), a recommender that replaces the dot product with a small neural network, we will use HesScale's Gauss-Newton variant or Sophia's pseudo-label estimator, refreshed every $k$ steps. The curvature is accumulated with the same propensity weights as the gradient and floored at $\gamma$.

In the multi-model DR schemes, each sub-model is preconditioned by its own weighted curvature. This also addresses the step-size mismatch between the prediction, imputation, and propensity models that cyclic training suffers from.

The third component is the empirical test of bias and variance separately. This requires semi-synthetic data with known ground truth, followed by confirmation on the real benchmarks that have randomized test sets.

Baselines

The naive reference point is the naive estimator, unweighted training on observed ratings, together with plain IPS and DR trained with SGD and with Adam. The Adam versions are what practitioners actually run. Every optimizer gets the same per-estimator learning-rate and regularization grid and the same early-stopping rule, and runs for the same number of gradient evaluations. Our method's extra curvature passes are counted against its budget.

The strongest existing variance-control method on the reported benchmarks is StableDR. On Yahoo! R3 with an NCF base model, StableDR reports an MSE of 0.1664 against 0.1705 for standard DR. On KuaiRec the strongest reported number is from UIDR in Relaxing the Accurate Imputation Assumption: AUC 0.804, compared with 0.745 for standard DR and 0.788 for ESCM2-DR. On Coat, the Doubly Calibrated Estimator reports an 8.37% MSE improvement over standard DR, but the file gives no absolute value. We found no reported numeric result on Lacuna for Cali-MR on any of the three datasets.

We will rerun StableDR and UIDR with their published designs under our tuning budget rather than copying their numbers. The decisive comparison is plain DR plus curvature normalization against StableDR with Adam. Beating it means lower MSE on Yahoo! R3, or AUC at least matching UIDR on KuaiRec, without any propensity constraint. Adding curvature normalization on top of StableDR and UIDR tests whether the gains stack.

The variance-reduction heuristics are run as optimizer-agnostic baselines. These are clipped IPS and DR with a tuned clipping cap, and global self-normalized IPS, which divides by the total weight across the whole dataset.

Two optimizer competitors are run as published. Sophia-H, using random-probe (Hutchinson) diagonal estimates, tests whether noisy curvature suffices. AGD, which estimates curvature from successive gradient differences, tests whether passive curvature suffices.

Three ablations isolate the key ingredient. Unweighted curvature preconditioning keeps the preconditioner but drops the weights from $h_j$, which removes the self-normalization. A weighted-count learning rate divides each item's update by $\lambda$ plus the summed weights of its ratings, dropping the factor $(\partial f/\partial\theta_j)^2$. Linear Adam modifies Adam to accumulate $|w g|$ instead of $w^2 g^2$.

Why not just…?

Why not just clip the propensity weights? Clipping changes the objective's minimizer. StableDR's experiments report that conventional weight handling either collapses or introduces bias when propensities are small. Our method leaves the minimizer unchanged by construction. The semi-synthetic sweep below separates the two effects by measuring bias and seed-to-seed variance independently as propensity skew grows. If tuned clipping matches us on both at every skew level, the proposal has failed.

Why not just use Adam, which already normalizes per coordinate? This is the most serious objection, because Adam is also scale-invariant under a uniform reweighting. The difference lies in how each normalizer treats weight dispersion. Adam normalizes by the second moment of the weights, while curvature normalizes by the first. HesScale independently shows that the squared gradient is a poor estimate of the true diagonal, though only in unweighted settings. The linear-Adam ablation tests this directly. If linear Adam recovers all of our gain, the contribution reduces to "accumulate first moments," which is still useful but much smaller, and we would report it that way.

Why not just use per-item learning rates proportional to inverse weighted counts? For matrix factorization item embeddings, this is close to our method minus the user-factor term, and it could plausibly work as well. That is why it is an explicit ablation. Two things would distinguish them. First, user-factor heterogeneity should matter when user embedding norms vary widely. Second, count-based rates have no analogue in the hidden layers of an NCF model. If the count-based rule ties our method on matrix factorization, we will say so, and the claim will rest on the NCF and DR multi-model results.

Why not just use global self-normalized IPS? Global normalization divides by the dataset's total weight, so it does not change the relative influence of one extreme rating on its own item's parameters. Per-coordinate normalization does. We expect global self-normalized IPS to help aggregate stability but not tail-item error. Tail-bucketed metrics will show whether that expectation holds.

Proposed Experiments

The core experiment uses MovieLens 100K with injected missing-not-at-random selection, following the semi-synthetic protocol of Relaxing the Accurate Imputation Assumption. In that paper, User-DR reaches a relative error of 0.006 against 0.047 for DR under user-specific bias. We will vary a single skew parameter that controls the minimum propensity over about two orders of magnitude. Because full ratings are known in this setup, we can measure each method's bias as the expected error of the fitted model against the true matrix, and its variance across 20 seeds. We will also corrupt the propensities with controlled miscalibration, since Cali-MR argues that overconfident propensities are the main failure mode. The proposal predicts that curvature normalization's variance falls below Adam's by a margin that grows with skew, and that its bias stays at the unclipped level.

On the real benchmarks, we will use Coat, Yahoo! R3, and the KuaiRec small matrix as randomized test sets. We will report MSE, AUC, and NDCG, broken down by item-popularity quintile. The per-coordinate effective sample size will be logged and correlated with per-item gains.

Two checks decide the claim. First, the per-item improvement over Adam should concentrate on items with low effective sample size. If gains are uniform across popularity, the mechanism is not self-normalization. Second, when all optimizers are run to full convergence with tuned regularization, the fixed-point argument predicts that the methods' bias converges. Any remaining gap then measures the finite-time and nonconvex-trajectory effects that real early-stopped training actually experiences.

Wall-clock overhead relative to AdamW will be reported against Sophia's roughly 5% and AGD's 7%, the overheads those papers report in their own settings.