Keeping Dormant Interests Alive in Multi-Interest Recommenders

Recommender systems increasingly describe each user with several vectors instead of one, one per "interest" (cookbooks, science fiction, running shoes), because a single vector lands on a useless average of unrelated tastes. Two problems then collide. The first is collapse: during training the interest vectors tend to merge into near-copies of each other, or all of a user's history gets assigned to one interest while the others sit empty. The second is drift: tastes change over time, so models weight recent behavior more heavily. This proposal argues that the usual fix for drift makes collapse worse in one specific way. When recent items dominate the training signal, the interests a user has not touched lately lose their gradient and are absorbed by the active ones. A model can then serve today's taste well and still fail when the user returns to an old one.

The core idea is to separate two time scales that current methods tie together. The weighting that decides which interest is relevant right now should be recency-biased. The regularizer that decides whether each interest slot stays occupied and distinct should look at the user's whole history. Concretely, we train end to end with an entropy objective on how history items are assigned to interest slots. Each item should be assigned confidently to one slot, and slot occupancy computed over the full history should not collapse. A separate learned, time-aware weighting handles relevance at prediction time. We also introduce an evaluation slice that current papers do not report: accuracy on next items that belong to an interest the user has left dormant for a long time.

Core Claim

Existing work handles the two halves separately. Every Preference Changes Differently: Neural Multi-Interest Preference Model with Temporal Dynamics for Recommendation and its companion Everyone's Preference Changes Differently: Weighted Multi-Interest Retrieval Model, together called MIP, learn a separate temporal weight for each interest cluster. They form those clusters with offline K-Means or Ward clustering and need a two-stage schedule, with weights frozen at 1 before being unfrozen, to stop clusters from degrading early in training. Multi-Tower Multi-Interest Recommendation with User Representation Repel, called MTMI, prevents interest vectors from collapsing by pushing them apart pairwise. It does this with distance-scaling hyperparameters and no temporal model. Controllable Multi-Interest Framework for Recommendation, called ComiRec, extracts interests by capsule routing or multi-head attention. It has no collapse term on the assignment probabilities and treats all history as equally recent.

Our claim is that one training objective can replace MIP's offline clustering and staged training, provided its collapse-prevention term is computed on a different time scale from its relevance term. Doing so should improve recommendations when users return to dormant interests without costing accuracy on current interests. The naive merge, marginal entropy computed under the same recency weights used for prediction, should fail in a predictable way. It pushes recent items to spread across all slots, which splits the current interest and overwrites the dormant ones. That failure is the central ablation.

Mechanism

For a user with history items $j = 1, \dots, n$ at timestamps $t_j$, an attention module produces $K$ interest vectors and a soft assignment $p(k \mid j)$ of each item to slot $k$. The auxiliary objective is

$$ \mathcal{L}_{\text{occ}} = \alpha \, \frac{1}{n}\sum_{j=1}^{n} H\big(p(\cdot \mid j)\big) \;-\; \beta \, H\big(\bar q\big), \qquad \bar q_k = \frac{1}{n}\sum_{j=1}^{n} p(k \mid j), $$

where $H$ is Shannon entropy, $\alpha$ and $\beta$ are weights, and $\bar q$ is slot occupancy averaged uniformly over the entire history.

The first term makes assignments sharp. If two slots were identical, items would split evenly between them and this term would be high, so minimizing it pushes slot vectors apart without a pairwise distance heuristic. The second term keeps slots populated. Because it is computed without recency weights, a slot that holds a user's heavy reading of cookbooks three years ago stays occupied even though the recency-weighted loss barely touches it.

Prediction is handled separately. A small time-aware network, in the spirit of MIP's cluster weight module, takes each slot's vector and the timestamps of its assigned items and outputs a relevance weight $w_k(T)$ at the current time $T$. The main next-item likelihood loss uses these weights. So a dormant slot can have near-zero relevance now while the regularizer protects its content.

A uniform occupancy target conflicts with users whose interests are truly unequal in size, the "task structure mismatch" the entropy-regularization literature warns about. Because of that, we also test a hinge variant that penalizes only slots whose full-history occupancy falls below a floor. This pressure only prevents empty slots and does not force equal ones.

The regularization literature both motivates and constrains this design. Stable Deep Reinforcement Learning via Isotropic Gaussian Representations shows that shifting targets drive representation collapse. It counters this with SIGReg, a regularizer that pushes the whole feature distribution toward an isotropic Gaussian, that is, a single round cloud. That paper motivates treating drift and collapse as one problem. Its global, single-peaked target is exactly what a multi-interest user violates, which is why we regularize discrete per-user assignments rather than global geometry. Self-Supervised Learning via Maximum Entropy Coding and MVEB: Self-Supervised Learning with Multi-View Entropy Bottleneck show that explicit entropy terms can replace architectural tricks for preventing collapse. We reuse that principle for clustering assignments. Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift, called NS-DPO, applies one global exponential discount to preference data in language-model alignment. Its acknowledged weakness is that a single discount cannot tell persistent preferences from transient ones. Our decoupling targets that weakness at the level of individual interests.

Research Program

We will build the model on a standard multi-head-attention multi-interest encoder, as in ComiRec-SA, trained with sampled-softmax next-item likelihood. We will add the occupancy objective and the time-aware relevance head, and drop offline clustering and staged training entirely.

The analysis tracks per-user collapse directly. We apply the Information Abundance measure from On the Embedding Collapse when Scaling up Recommendation Models, the sum of singular values divided by the largest, to each user's $K \times d$ matrix of interest vectors. We also count the number of slots with occupancy above a small threshold. Both are logged across training and across a user's timeline. The hypothesis is mechanistic. Under recency-weighted training without the regularizer, slots belonging to dormant interests should drift toward the active slots, which shows up as falling Information Abundance and rising cosine similarity between slots. The full-history occupancy term should stop this.

The evaluation contribution is a reactivation slice. On datasets with item categories (Amazon Books, Taobao), we tag each held-out target item as current, reactivated, or novel. A current item's category appears among the user's last $m$ interactions. A reactivated item's category appeared earlier but not in the last $m$. A novel item's category never appeared in the history. We report Recall@50 and NDCG on each slice. Reactivation is where our claim lives, and aggregate Recall averages it away.

To test the mechanism in a setting where we know the answer, we also build a controlled drift benchmark from MovieLens-20M genres. We resample real user histories so that each synthetic user has known interests with scheduled dormancy lengths and return times. We can then measure reactivation recall as a function of dormancy length.

Baselines

The naive reference is a single-vector sequential model, TiSASRec, which MIP reports as its strongest non-multi-interest baseline. Next to it is ComiRec-SA with no temporal modeling and no collapse term. Together these show what multi-interest structure alone buys.

The strongest existing method is MIP. It is the only system in the file that combines multiple interests with a learned temporal weight per interest. It reports Recall@50 of 0.789 on Amazon Books against 0.667 for TiSASRec (Every Preference Changes Differently). On the industrial Pinterest data it reports Recall@20 of 0.353 against 0.296 for PinnerSage. We cannot access Pinterest, so Amazon Books, Taobao, and MovieLens-20M are the comparison sets. Because the 0.789 figure comes from MIP's own split and candidate protocol, we will re-run MIP, with its two-stage schedule and both clustering choices, inside our pipeline. It gets the same splits, history length, embedding size, number of interests $K$, and a matched hyperparameter search budget. We report our reproduced number next to the published one.

MTMI is the strongest collapse-prevention competitor. It is run with its IDWLoss and a search over its distance-scaling and divergence parameters equal in size to our search over $\alpha$ and $\beta$. For MTMI and for Taobao, no absolute numbers are reported in the file, so the reproduced numbers are the only reference.

The key ablation replaces full-history occupancy $\bar q$ with occupancy computed under the same recency weights as the prediction loss. This isolates the time-scale decoupling. A second ablation removes the occupancy term entirely while keeping the time-aware head.

Beating MIP means a statistically significant gain in reactivation-slice Recall@50, measured over five seeds, while aggregate and current-slice recall stay within noise of MIP or better. A gain on reactivation paid for with a drop in current-interest recall would not count as a win.

Why not just…?

Why not just apply exponential time decay to fixed clusters? MIP's own ablation compared learned per-cluster temporal weights with a heuristic exponential decay and with equal weights. The learned weights won on every public dataset. NS-DPO's analysis likewise shows that a single discount factor requires knowing the total amount of drift in advance. We therefore keep learned relevance weights. Decay addresses relevance, not occupancy, so it cannot stop a dormant slot from being absorbed. We include it only as a weak reference.

Why not just use MTMI-style pairwise repulsion? Repulsion keeps vectors apart in space. It does not keep a slot tied to the items that defined it, so a pushed-away dormant slot can drift into empty space and still fail at reactivation. MTMI also depends on tuned distance parameters, and MIP reports that a triplet-style separation loss underperformed plain likelihood. This alternative is settled empirically by the MTMI baseline on the reactivation slice. If MTMI matches us there, the entropy formulation adds little, and we will say so.

Why not just stop gradients to, or freeze, slots whose relevance weight is near zero? This is the most serious threat, because it directly protects dormant content with almost no machinery. Its weakness is that freezing requires a threshold for "dormant" and blocks legitimate updates when an old interest evolves, for example a user moving from beginner to advanced cookbooks. It also does nothing about collapse among active slots. We run it as a baseline with a tuned threshold. If it matches our reactivation recall, the proposal's contribution shrinks to removing MIP's offline clustering and staged training, and we will report it that way.

Why not just use plain marginal entropy or balanced assignment on recent history, or simply lengthen the history window and increase $K$? Plain recency-weighted marginal entropy is our key ablation. We predict it splits the current interest across slots, which should be visible as lower current-slice recall and lower per-slot purity. Longer windows and larger $K$ are swept for all methods. If MIP with a long window closes the reactivation gap on its own, then collapse was not the bottleneck, and the controlled benchmark will show whether the gap depends on dormancy length.

Decisive Checks

The idea is supported only if three things hold together.

First, on the controlled MovieLens drift benchmark, reactivation recall for recency-trained baselines should fall as dormancy length grows. Our model should flatten that curve, and the recency-weighted-entropy ablation should not.

Second, per-user Information Abundance and slot occupancy should stay stable for our model while falling for the no-regularizer ablation. This would tie the accuracy gain to preventing collapse rather than to extra capacity.

Third, a sweep of the window over which occupancy is computed, from the last $m$ items up to the full history, should show reactivation recall rising with window length while current-slice recall stays flat. That dose-response pattern is the direct test of the time-scale decoupling.

If the frozen-slot baseline matches us on all three, or if the reactivation slice shows no gap between MIP and TiSASRec in the first place, the premise fails and we will report it as a negative result.