Content-Conditioned Priors for Streaming Bayesian Tensor Factorization

Streaming Bayesian tensor factorization is a family of methods that learn low-dimensional user, item and context vectors from a stream of interaction batches. These methods never store the full data. Instead they keep small running sums, called sufficient statistics, that summarize everything seen so far. They scale to billions of cells and update in closed form, but they have a blind spot: an entity with no interactions gets no sufficient statistics, so its vector stays at a generic prior. That prior is identical for every newcomer and says nothing about that particular user or item. Cold-start recommendation, the problem of serving brand-new users or items, has its own tools: content encoders, meta-learning, and contrastive alignment. Those tools are trained offline in episodes or epochs, and they give point estimates with no measure of how unsure the model is.

This note proposes to close that gap in one step. The prior over each entity's latent vector becomes a Bayesian linear regression on that entity's content features, such as frozen text, image or audio embeddings, or demographic attributes. The regression weights are themselves learned from running sufficient statistics gathered from warm entities. A new item then enters the stream with an informed, calibrated prior. Its first few interactions update that prior exactly, through the same closed-form rules the tensor model already uses. In this design, "fast adaptation from 5 to 15 interactions", the selling point of meta-learning, reduces to a conjugate Gaussian posterior update. It needs no episodes, no inner loop and no stored history.

Core Claim

Hierarchical priors that depend on side information are an old idea. What is missing in the streaming tensor literature assembled here is a version whose content-to-factor map is updated from the same recursive statistics as the factors, and that therefore stays single-pass. We claim three things. First, with a content-conditioned prior whose parameters are tracked by online expectation-maximization (EM), a streaming Pólya-Gamma tensor model can serve zero-history entities competitively with offline cold-start methods. Second, it improves steadily as the first interactions arrive, without retraining. Third, its posterior variance for cold entities is well calibrated, which offline point-estimate methods cannot provide.

Where This Sits

The mechanism comes from three streaming Bayesian tensor models. Scalable Probabilistic Tensor Factorization for Binary and Count Data, called PGCP, uses Pólya-Gamma augmentation to turn logistic and negative-binomial likelihoods into Gaussian-like forms, and runs online EM over mini-batch statistics. Its authors list side information as future work. Scalable Bayesian Non-Negative Tensor Factorization for Massive Count Data, called BNBCP, obtains cost that scales with the number of observed entries, not the full tensor size, for overdispersed counts (counts whose variance exceeds their mean). Streaming Bayesian Deep Tensor Factorization, called SBDT, replaces noisy gradients with deterministic moment matching. It reports that gradient-based streaming variational Bayes is erratic when there is no fixed validation set. All three motivate the proposal, because in each of them a new entity's factor equals the prior. SBDT also constrains the design: its stability argument is why we keep updates closed form rather than bolting a neural content encoder onto gradient descent.

The cold-start side supplies the competitors and the evaluation regimes. MAMO retrieves a personalized starting point from memories indexed by user demographics, then adapts it through bi-level meta-optimization. CLCRec maximizes mutual information between content and collaborative embeddings, and during training randomly swaps the collaborative embedding for the content one. Both are offline and deterministic. Our proposal keeps MAMO's intuition that the starting point should depend on side information, and CLCRec's intuition that content and collaborative spaces should be tied together. It replaces both learning procedures with conjugate streaming updates. SimGCL shows that contrastive gains in sparse warm settings come mostly from spreading embeddings uniformly over the unit sphere. We treat it as the strongest warm-regime reference. Adding contrastive uniformity terms to the closed-form model is out of scope; the research file identifies that as a separate unsolved problem.

Model

Let $x_i \in \mathbb{R}^p$ be a fixed content feature vector for entity $i$, taken from a frozen pretrained encoder or from one-hot attributes. Let $u_i \in \mathbb{R}^R$ be its latent factor in one tensor mode. We place the prior

$$u_i \sim \mathcal{N}(W^\top x_i,\ \Lambda^{-1}),$$

where $W \in \mathbb{R}^{p \times R}$ is a mode-specific map from content to factors, and $\Lambda$ is a diagonal precision. $\Lambda$ keeps PGCP's multiplicative gamma process shrinkage, a prior that pushes unneeded latent dimensions toward zero so the model can choose its own rank. Online EM maintains the running averages $S_{xx} = \mathbb{E}[x_i x_i^\top]$ and $S_{xu} = \mathbb{E}[x_i\, \mathbb{E}[u_i]^\top]$ over entities touched in each batch. These averages are decayed with the Robbins–Monro step sizes already used by online EM. The M-step is ridge regression, $W = (S_{xx} + \gamma I)^{-1} S_{xu}$, whose cost is $O(p^2 R)$ regardless of catalog size.

Given this prior, an entity with $k$ observed interactions has a closed-form Gaussian posterior. Its precision is

$$\Lambda + \sum_{n=1}^{k} \omega_n z_n z_n^\top,$$

where $\omega_n$ is the expected Pólya-Gamma variable for interaction $n$, and $z_n$ is the elementwise product of the other modes' factors for that entry. For Gaussian ratings, $\omega_n$ is simply the noise precision. The posterior mean moves from $W^\top x_i$ toward the collaborative evidence at a rate set by how informative each interaction is. When $k=0$ the model returns the content prediction, and when $k$ is large it returns the collaborative factor. Row indices are allocated when a new entity arrives, which removes the fixed-cardinality restriction of the original accumulators.

Two design questions will be studied explicitly. The first is whether $W$ should be fit to posterior means alone or also absorb posterior covariance through an expected-sufficient-statistic correction, which matters when most warm entities are themselves sparse. The second is how the shrinkage prior on $\Lambda$ interacts with the content map. A dimension that the collaborative signal prunes can still carry content information, and we will measure whether pruning erases content-predictable directions.

Research Program

We will implement the model on top of PGCP-style online EM for binary data, BNBCP-style augmentation for counts, and Gaussian likelihoods for ratings, using the same streaming loop throughout. The analysis has three parts.

The first part is a cold-to-warm trajectory. Entities are revealed in timestamp order. Each cold entity is scored at $k = 0, 1, 2, 5, 10, 15$ interactions without any global retraining.

The second part is calibration. We check whether posterior predictive variance for cold entities tracks realized error, using expected calibration error for binary data and negative log-likelihood for all likelihoods. As a downstream use of the variance, we will also run Thompson-sampling selection on replayed logs.

The third part is higher-order context. On multi-behavior logs we model user × item × behavior as a three-way tensor, and ask whether a cold item's content prior transfers across behavior types through the shared item mode.

Baselines

The naive reference is the same streaming PGCP or BNBCP model with its original zero-mean prior. New entities get only popularity-level scores until they accumulate data. Any gain over it measures what the content prior contributes. To beat it, the proposed model should score clearly higher at $k=0$ and still lead at $k=5$.

On the user side, the strongest existing cold-start method with a reported number is MAMO. MAMO reports an MAE of 0.8894 on MovieLens-1M in the cold-user, cold-item setting, against 0.9299 for MeLU. We will reproduce MAMO on its own splits with the same demographic features supplied to our $x_i$. Both methods will get the same support-set sizes and the same hyperparameter search budget. Our model is allowed a single pass over the training stream, while MAMO gets its usual episodic training. Matching 0.8894 in a single pass would count as parity. Beating it by a margin larger than the seed-to-seed spread would count as a win.

On the item side, the strongest competitor is CLCRec, which reports relative cold-item Recall@10 gains of 96.24% on Amazon and 103.95% on TikTok over its baselines. No absolute Recall@10 value appears on Lacuna for CLCRec on these datasets, so we will rerun it with identical frozen multimodal features and identical cold-item splits. We will use its reported best setting of about 256 negatives. Beating CLCRec means higher cold-item Recall@10 at $k=0$, or equal Recall@10 together with better calibration and a faster $k=1$–$5$ improvement without retraining.

For warm-regime sanity on Amazon-Book, the reference is SimGCL, which reports more than a 25% Recall@20 gain over LightGCN. We do not expect to beat it on warm items. The requirement is only that the model stays within a stated tolerance of LightGCN, so that the cold-start gains do not come at an unreported warm-start cost.

Two ablations isolate the key ingredient. The first is post-hoc regression: train the zero-prior streaming model, then fit $W$ once by ridge regression from content to the learned factors. The second freezes $W$ after the first 10% of the stream. Together they separate joint coupling of the content map and the factors from mere access to content, and a continuously updated map from a static one.

Why not just…?

Why not just fine-tune embeddings with Adam and SimGCL noise as data arrives? This is the cheapest option in engineering terms. Its weakness in our setting is specific: a zero-history item receives no gradient, so it needs a content encoder anyway. Once one is added, the approach becomes a gradient-trained CLCRec variant. SBDT's evidence that gradient-based streaming inference is unstable without a validation set suggests it will fluctuate across the stream, and we will test that directly. We will run this streaming SimGCL-plus-content variant and plot cold-item Recall@10 over stream time for both methods.

Why not just use CLCRec's content-to-collaborative MLP, trained offline? This is the real threat. A nonlinear encoder may simply map content better than our linear map on frozen features. We accept that CLCRec may win at $k=0$ on TikTok and Kwai. The proposal still earns its place if three things hold: our model overtakes CLCRec within the first few interactions, it does so without retraining, and it provides calibrated uncertainty. If CLCRec is better at every $k$ and our calibration adds nothing measurable to Thompson-sampling reward, the claim fails. We will report that outcome as a negative result.

Why not just use MAMO? MAMO requires demographic profiles to index its memories, and it requires episodic offline training. Our model degrades gracefully when profiles are missing. We will test this by rerunning the MovieLens-1M and Book-Crossing comparison with demographics removed and item content kept, where MAMO's profile memory has nothing to index. If MAMO still wins with profiles present, we will say so. The contribution then narrows to the streaming, profile-free regime.

Why not just regress content onto learned factors after training? If this post-hoc ablation matches the full model, the joint streaming coupling is unnecessary and the idea reduces to a known two-stage recipe. We expect the joint model to help mainly for items with 1–10 interactions. For those items, the content prior should regularize noisy factors, and the factors should in turn sharpen $W$. The sparsity-bin breakdown described below is designed to show whether that happens.

Proposed Experiments

The datasets follow the research file. For cold users we use MovieLens-1M and Book-Crossing with MAMO's four warm/cold splits. For cold items we use TikTok, Kwai and Amazon, whose multimodal features come from CLCRec's setup. For multi-behavior three-way tensors we use Tmall, IJCAI-Contest and Retail Rocket, where CML reports 25.4% HR@10 and 25.8% NDCG@10 gains over its strongest baseline, again only as relative figures. For scale, we use the 117,054 × 438 × 67,095 Transactions count tensor, holding out newly appearing households and items. Metrics are MAE for ratings, Recall@10 and 20, HR@10 and NDCG@10 for ranking, held-out log-likelihood, expected calibration error, and wall-clock time and memory per batch.

Three results decide the claim. The first is the $k$-trajectory curve against CLCRec and MAMO, on which the proposal must win at some small $k>0$ without retraining. The second is the post-hoc-regression ablation broken down by item sparsity bin, which must show that joint streaming updates beat two-stage fitting for items with between 1 and 10 interactions. The third is calibration: posterior variance at $k=0$ must rank-correlate with realized cold-entity error and improve replayed Thompson-sampling reward over a variance-free version of the same model. If all three hold, streaming Bayesian tensor models gain a principled cold-start pathway. If the first or second fails, we learn that content priors in closed-form streaming models are dominated by simpler offline encoders, which is also worth knowing.