Does Dwell-Time Supervision Move the Point Where Recurrent Recommenders Start to Win?

Sequence recommenders face a known dilemma about model size. On large interaction logs, recurrent encoders do best. These are networks that read a user's history one step at a time and carry a running memory vector. On small or sparse logs, simple averaging or a shallow multi-layer perceptron (MLP, a stack of plain fully connected layers) beats them. The data volume at which the ranking flips is called the complexity crossover point. It has been measured for language models and observed in recommendation, but it is always treated as a fixed property of the data size.

This proposal tests whether the crossover is instead a property of how much supervision each sequence carries. Engagement logs record more than clicks. They also record how long a user stayed on each clicked item, the dwell time, which gives a continuous label at almost every step. Our hypothesis is that training a recurrent encoder to predict dwell time alongside the next click moves its crossover to smaller datasets. If it does, the extra signal itself is doing the work, not generic regularization and not dwell time used as an input feature. In practice this would give recommender builders a rule for when a recurrent backbone pays off on a sparse product.

Core Claim

The prior evidence supplies two facts that have not been put together. On the motivating side, Neural network language models for low resource languages measured an explicit crossover: a feed-forward language model beat a recurrent one at about 100k tokens, and the recurrent model only won past 500k tokens. In recommendation, State Encoders in Reinforcement Learning for Recommendation found that GRU and attention encoders were best on Yahoo! R3 (15,400 users). On Coat (290 users), a one-layer MLP beat them and the attention encoder fell below simple averaging. A GRU is the gated recurrent unit, a recurrent cell with learned gates that decide what to keep in memory.

Separately, the joint click-and-dwell models JUMP and HyperNews report that the duration task helps click prediction. In HyperNews, an ablation shows the two-task model beating a CTR-only model, where CTR is click-through rate. Neither paper varies data size, and neither compares against a simpler encoder given the same extra supervision. So no one knows whether dwell supervision rescues recurrent encoders specifically in the small-data regime, or whether it helps every encoder equally and leaves the crossover where it was.

The claim is falsifiable. Let $N^*_{\text{single}}$ be the training-set size at which a recurrent encoder first beats the best MLP-family encoder under click-only training. Let $N^*_{\text{multi}}$ be the same point when both encoders are trained with the dwell objective. We predict $N^*_{\text{multi}} < N^*_{\text{single}}$, meaning the recurrent encoder starts winning on less data.

We also predict where the effect comes from. It should disappear when dwell targets are shuffled within each user. Shuffling keeps the amount of supervision and its scale but destroys its alignment with the sequence. We further predict that the shift is larger for gated recurrence than for averaging. The reason is that dwell time is a sequential quantity. It depends on where the user is in a session, which is exactly the information a running memory can use and a pooled average cannot.

Research Program

We fix one training objective and vary three things: the encoder, the data size, and the form of the dwell signal. Every model predicts the next item with a softmax ranking loss. In multi-task runs it also predicts the log dwell time of the current item through a censored log-normal likelihood. This follows JUMP's observation that log-dwell is roughly Gaussian and that session-ending clicks have no departure time.

The combined loss is

$$\mathcal{L} = \mathcal{L}_{\text{item}} + \lambda \sum_t \Big[ \delta_t \log \phi\big(\tfrac{\log \tau_t - \mu_t}{\sigma}\big) + (1-\delta_t)\log\big(1-\Phi\big(\tfrac{\log \tau_t - \mu_t}{\sigma}\big)\big) \Big]$$

The symbols are defined as follows:

  • $\tau_t$ is the dwell time at step $t$.
  • $\delta_t$ equals 1 if the dwell time was observed and 0 if it was censored at session end.
  • $\mu_t$ is the predicted mean log-dwell, computed by the dwell head from the encoder state.
  • $\sigma$ is a learned scale.
  • $\phi$ and $\Phi$ are the standard normal density and cumulative distribution.
  • $\lambda$ weights the dwell task and is tuned per run.

We keep the dwell head linear, as JUMP does, so that capacity differences live only in the encoder.

The encoder family spans the complexity range. The low end is mean pooling and a one-layer MLP over pooled embeddings. Next comes a causally masked MLP in the style of TriMLP, which showed that MLPs need triangular masking to avoid leaking future positions into past ones. Above that is a linear recurrent unit as in LRURec. That encoder removes nonlinearity from the memory update, constrains the update to be stable, and trains in parallel. The high end is a GRU.

We include the linear recurrent unit because it splits "recurrent" into two separate ingredients: persistent ordered state and nonlinear gating. If dwell supervision helps the GRU but not the linear unit, the gates are what matter.

For each dataset we subsample users at log-spaced fractions, from a few hundred users up to the full set, with five seeds per point. From the resulting learning curves we estimate each crossover by interpolating where the curves for the two encoder classes intersect, and we bootstrap over seeds to get confidence intervals. The analysis also checks for the failure mode ORCA identified on fixed multi-task backbones: dwell predictions collapse to the extremes and under-represent moderate durations. We will report the histogram of predicted dwell per encoder and data size. That shows whether recurrent state reduces this collapse or whether it worsens at small data, which would work against our hypothesis.

Baselines

The naive reference point is a popularity ranker for items paired with the per-user mean log-dwell for duration. It runs on identical splits and costs nothing to tune. Every learned model must beat it at every data size we report. A model that does not is excluded from the crossover estimate at that size.

The strongest simple competitor is the click-only one-layer MLP over pooled history. Huang et al. found this design wins at small scale, and Reweighting Clicks with Dwell Time in Recommendation found a shared-bottom MLP matched a mixture-of-experts model in a 29.7M-user deployment. We also run the MLP with the dwell head attached. This matters because our claim concerns the gap between encoders, not whether dwell helps in absolute terms. Each model gets the same tuning budget: a fixed number of random-search trials over width, dropout, weight decay, $\lambda$, and learning rate. The same budget applies at every data size.

The strongest existing recurrent multi-task method is JUMP. On RecSys15 it reports Recall@20 of 0.7168 against NARM's 0.6907. On CIKM16 it reports a dwell mean absolute error of 180.6 against ATRP's 229.8. We will reimplement JUMP's fast-slow recurrent encoder as an additional high-capacity point. Our full-data GRU-plus-dwell runs should land near these numbers to show the reimplementation is sound. Matching or beating JUMP at full data is not the goal. The goal is to show its family wins at smaller data than it does without dwell.

For the multi-behavior, sequence-model-free side, BMLP reports HR@10 of 0.426 on Tmall against SASRec's 0.369. Tmall lacks dwell time, though, so it serves only as a sanity check that our MLP encoders are competitive. For dwell-related metrics on Tenrec, ORCA reports only an average 10.6% uplift, not absolute values. We found no reported absolute dwell result on Lacuna for Tenrec.

The key ablation replaces real dwell targets with targets shuffled within each user. This keeps the label scale and density but removes the alignment between dwell and sequence position. If the crossover shift survives shuffling, our explanation is wrong.

Why not just…?

Why not just regularize the GRU harder instead of adding a task? Dropout, weight decay, and a smaller hidden size also cut variance. A crossover shift could therefore come from any capacity control, not from engagement information. This is a real threat. We handle it by giving the click-only GRU the same tuning budget over exactly these regularizers at every data size. We also add a non-engagement auxiliary task with the same per-step density, namely predicting the next item's category, which SGRec-style work uses as a regularizer. Our claim survives only if real dwell moves the crossover further than both tuned regularization and the category task.

Why not just feed dwell time in as an input feature? JUMP already embeds dwell jointly with item IDs. If dwell-as-input gives the recurrent encoder the same small-data advantage, then the supervision itself adds nothing. We run a dwell-as-input, click-only arm for every encoder and report its crossover alongside the others. If it matches the multi-task arm, we will say so and restate the finding as "dwell information" rather than "dwell supervision".

Why not just filter out short-dwell clicks and train a single-task model? Xie et al. show that thresholds near 15 seconds remove clickbait well in production. Filtering, however, shrinks the already small training set. It also discards the censored session-ending clicks that the likelihood above keeps. We include a filtered click-only arm at each data size. If filtering matches multi-task training at the small end, that is the cheaper recommendation and we will report it as such.

Why not just use an MLP everywhere and skip recurrence? If the MLP plus dwell head dominates at every data size we can afford, then the crossover moves off the chart and the practical answer is to never use recurrence. That would be a useful negative result. We count it as a possible outcome, not a failure of the study.

Proposed Experiments

The primary datasets are the ones in the file that carry dwell or active time:

  • CIKM16 and RecSys15, both used by JUMP.
  • Adressa-1week, used by HyperNews; that paper reports AUC 0.8661 against DAN's 0.8234 for the click task.
  • Tenrec QK-article and QB-article from Tenrec, which record read time.

Metrics are Recall@20 and NDCG@20 for items. For dwell we use mean absolute error and the censored log-likelihood. We also report the share of predictions falling in moderate-duration bins, ORCA's diagnostic.

The decisive check is the bootstrap interval on $N^*_{\text{single}} - N^*_{\text{multi}}$ for the GRU-versus-best-MLP pair on each dataset. Two further checks support it. The first is whether that gap shrinks to zero under shuffled dwell. The second is whether it is larger for the GRU than for the linear recurrent unit.

The hypothesis is supported if the gap is positive on at least three of the five datasets and vanishes under shuffling. It is refuted if the gap is zero or negative. It is also refuted, in its supervision form, if dwell-as-input or tuned regularization produces the same shift. A secondary result reports how the size of the shift relates to how predictable dwell is from the click sequence in each dataset. That relation would give a first, empirically grounded rule for when engagement labels make a recurrent backbone worth its cost.