Systems that tag a social-media post with emotions treat every post as if it arrived alone. A post that says "can't believe it's happening again" is ambiguous on its own. It is much less ambiguous if the same person wrote something grief-stricken two hours earlier, and ambiguous again if that earlier post was six months ago. We propose to carry an emotional state from one post to the next for each user. Between posts, that state fades back toward the user's own baseline at a rate that depends on the time elapsed. Each emotion gets its own learned fading rate: anger might fade within hours while sadness lingers for days. The task is multi-label emotion classification, meaning a post can carry several emotions at once (grief and anger and relief).
The bet is that much of what current systems lose on rare emotions is missing context rather than a lack of training examples. If so, a time-aware memory of recent posts should help rare labels more than loss reweighting does. The model should also produce something no current emotion classifier offers: an estimated "half-life" for each emotion, learned from data, which can be inspected, tested for stability, and used as a feature downstream.
Core Claim
The current recipe for imbalanced multi-label emotion detection has two parts. The first is a pretrained encoder, ideally one adapted to the text domain. The second is focal loss, a loss that shrinks the contribution of examples the model already gets right so that rare, hard ones dominate training. Focal-RoBERTa is the clearest instance: it reaches macro F1 0.7563 on SemEval-2025 Task 11 English. Its own error analysis, however, shows anger confused with sadness because the two share vocabulary. SpanEmo reports the same anger/disgust conflation.
These are confusions that the text of a single post cannot resolve, and reweighting cannot fix a confusion that the input does not determine. Our claim is that a user-level state with continuous-time decay and reversion to a baseline resolves part of this ambiguity. We test two consequences. First, gains should concentrate on rare and negative emotions. Second, the focusing strength that works best in focal loss should go down once the prior is present, with better probability calibration as a side effect.
No paper in the research file combines these pieces. Time-aware recurrent models (T-LSTM, Time-LSTM, GRU-M) handle clicks, problem steps, or sensor values, never text or emotions. Emotion systems (LEM, SpanEmo, Focal-RoBERTa, the multitask #MeToo model of Sawhney et al.) classify isolated posts. The one longitudinal mental-health system in the file, the CLPsych 2021 psychologically-informed-priors model, pools posts into fixed 30-day and 182-day windows instead of modelling the gaps between posts.
The Model
A domain-adapted encoder reads post $k$ of a user and outputs emotion logits $f(x_k) \in \mathbb{R}^E$, where $E$ is the number of emotions. We use BERTweet, which Sawhney et al. found better than BERT-base on informal disclosure text. The user also carries a state vector $s$. When post $k$ arrives after a gap $\Delta t_k$, the state first fades toward a baseline $\mu$:
$$s_k^- = \mu + (s_{k-1} - \mu)\odot \exp(-\lambda\,\Delta t_k), \qquad z_k = f(x_k) + A\,s_k^-$$
Here $\lambda \in \mathbb{R}_{\ge 0}^E$ holds one learned decay rate per emotion. The half-life of emotion $e$ is $\ln 2/\lambda_e$. The matrix $A$ lets a lingering state for one emotion shift the logits of others, which is where co-occurrence structure enters. $z_k$ are the final logits.
After the prediction, a small gated update folds the current post into the state, $s_k = \mathrm{GRU}(s_k^-, f(x_k))$. The baseline $\mu$ is a running mean of the user's past states, shrunk toward a population mean when the history is short. This shrinkage addresses the cold-start weakness that Time-LSTM reports for sparse histories.
The fade-to-baseline form borrows from GRU-M's argument that decaying only the last observation ignores the global mean, and from the exponential state discount systematized by García-Durán et al.. The model is strictly causal, with no look-ahead, so it avoids the streaming cost of GRU-M's bidirectional imputation. Each output is trained with a per-label focal term $-(1-p_t)^\gamma \log p_t$ from Lin et al.. The focusing strength $\gamma$ is swept rather than fixed, because its optimal value is one of our measurements.
Research Program
The first obstacle is data. The file contains no benchmark with multi-label emotion annotations on timestamped user timelines: SemEval-2018 Task 1, GoEmotions, and SemEval-2025 Task 11 are all post-level. We will build one.
We will sample timelines from sources that already carry user histories and timestamps. These are Reddit histories underlying RSDD-Time and the Twitter feeds of the CLPsych 2021 task, both under their data-use agreements. We will annotate roughly 300 users with about 30 consecutive posts each using the 11-emotion scheme of SemEval-2018 Task 1, so that SemEval-2018 can pretrain the encoder. Annotators will see posts in random order across users, so their labels cannot themselves encode the timeline context. Without that safeguard, a temporal model could "win" simply by recovering annotator context effects.
Using organic timelines also answers the warning from Bagdon et al.: their models dropped from 0.62 to 0.40 F1 when moved from study-elicited text to natural posts. Because this is mental-health data, the work will need ethics review and de-identification, and the half-life estimates will be reported only as population aggregates, never per user.
The analyses follow from the claim. We break per-emotion F1 down by label frequency and by gap length. We check whether learned half-lives are stable across seeds and data splits, and whether they are ordered plausibly, for example sadness above surprise. We trace how the best $\gamma$ and the expected calibration error move once the prior is added, connecting to evidence from Zhou et al., Dual Focal Loss, and AdaFocal that focal loss distorts confidence. Finally, we test whether emotion trajectories improve downstream suicide-risk prediction on CLPsych 2021.
Baselines
Naive reference. The do-nothing reference is the same BERTweet encoder applied to each post alone with plain binary cross-entropy. The Focal-RoBERTa paper reports that this kind of setup collapses to macro F1 0.14–0.17 on SemEval-2025 Task 11, so it bounds how much the loss alone matters.
Strongest existing method. The strongest competitor is the single-post Focal-RoBERTa recipe (encoder plus attention pooling plus focal loss with $\alpha=1,\gamma=2$), which reports macro F1 0.7563 on SemEval-2025 Task 11 English. It is the strongest competitor because it already addresses imbalance directly. If context adds nothing beyond it, our thesis fails.
We reimplement it on both encoders, RoBERTa and BERTweet, and run it in two places. On SemEval-2025 Task 11 we should match its reported number as a sanity check. On our timeline set, where no reported result exists anywhere in the file, it gets the same pretraining, the same splits by user, and the same hyperparameter budget, which includes sweeping $\gamma$ and $\alpha$. We also include Asymmetric Loss and SpanEmo as alternative single-post systems. SpanEmo reports micro F1 0.713 and Jaccard 0.601 on SemEval-2018 English.
What counts as beating it. On the timeline test users, we must improve macro F1 over the best tuned single-post system by a margin that exceeds variation across seeds and holds under bootstrap resampling over users. The improvement must be concentrated in the rarest third of emotions and must not come with worse calibration.
Ablations. Three ablations isolate the key ingredient. The first is a time-blind version with $\Delta t$ fixed to a constant, which keeps the memory but removes time. The second shuffles timestamps within each user. The third removes the fade toward the baseline, setting $\mu = 0$ so the state simply decays away as in T-LSTM-style decay. A fourth ablation swaps BERTweet for RoBERTa to measure how much the domain-adapted encoder matters relative to the temporal state.
Downstream baseline. For CLPsych 2021, the baselines are the reported regularized-regression AUC of 0.844 (182-day window) and 0.599 (30-day window). We compare against those by adding emotion-trajectory summaries to the same regression.
Why not just…?
Why not just feed the whole history to a transformer with $\Delta t$ as an input feature? This is the most serious threat, and we run it directly. The competitor is a long-context encoder over the last $n$ posts with log-gap embeddings, under the same compute budget. Our model has two structural advantages, and both are testable. It extrapolates to gap lengths rarely seen in training, because the exponential form is fixed and only its rate is learned. It also keeps a cost that is constant per post regardless of history length, whereas attention cost grows with the square of the context.
We will hold out the longest-gap transitions and the shortest histories. If the feature-input transformer matches us there too, the honest conclusion is that time matters but the decay structure does not. The paper would then report that result and recast the half-lives as a probing analysis of the transformer.
Why not just smooth per-post predictions with an exponential moving average after the fact? This needs no training, so it is a cheap test of whether learning is required. It cannot learn cross-emotion effects (the matrix $A$), cannot learn a different rate for each emotion, and cannot change what the encoder attends to. We include it with the decay rate tuned on validation data. If it captures most of the gain, the contribution shrinks to an annotation and analysis study, and we will say so.
Why not just use a better imbalance loss on single posts? Asymmetric or class-balanced losses (OFFLangOne, Sawhney et al.) target the same rare-label failure. A loss can only reweight evidence that is already in the input, though, and the anger/sadness confusions reported for Focal-RoBERTa stem from shared vocabulary within a single post. The comparison against the best tuned loss on single posts, broken down by emotion, settles this. Context should help exactly on the confusion pairs that reweighting leaves untouched.
Why not just use a domain-adapted encoder and stop there? BERTweet is already in every arm of the comparison, so any gain we report is on top of domain adaptation. The encoder-swap ablation shows how much each piece contributes.
Decisive Checks
Five results would decide the proposal. The first is a gain over the tuned single-post focal baseline on held-out users, concentrated in the rare emotions, that disappears when timestamps are shuffled or fixed to a constant. That pattern would show the model is using elapsed time, not just ordering.
The second follows from Ye et al., who found time gates help in self-paced settings and fade when a tutor imposes the pacing. The analogous prediction here is that gains grow with how irregular a user's posting is. A flat curve would count against the claim.
The third is learned half-lives that are consistent across seeds and data splits, with an ordering that is not arbitrary. The fourth is an optimal $\gamma$ that drops, and a calibration error that falls, once the prior is added. That would be evidence that part of focal loss's job was compensating for missing context.
The fifth is downstream. Emotion-trajectory features should add AUC above the reported 0.599 on CLPsych Subtask 1, where 30 days of dense recent posting is exactly the regime a decaying state should exploit. Subtask 2's 0.844 sets a harder bar, which we report either way.
References
- NITK-VITAL at SemEval-2025 Task 11: Focal-RoBERTa: Addressing Class Imbalance in Multi-Label Emotion Classification
- SpanEmo: Casting Multi-label Emotion Classification as Span-prediction
- What Time is It? Student Modeling Needs to Know
- What to Do Next: Modeling User Behaviors by Time-LSTM
- GRU-M: A Joint Impute and Learn Approach for Sequential Prediction under Missing Data
- Latent Emotion Memory for Multi-Label Emotion Classification
- Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures
- Using Psychologically-Informed Priors for Suicide Prediction in the CLPsych 2021 Shared Task
- Recursive Input and State Estimation: a General Framework for Learning from Time Series With Missing Data
- Focal Loss for Dense Object Detection
- RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses
- SemEval-2018 Task 1: Affect in Tweets
- Donate or Create? Comparing Data Collection Strategies for Emotion-labeled Multimodal Social Media Posts
- Revisiting Reweighted Risk for Calibration: AURC, Focal Loss, and Inverse Focal Loss
- Dual Focal Loss for Calibration
- AdaFocal: Calibration-aware Adaptive Focal Loss
- Asymmetric Loss For Multi-Label Classification
- OFFLangOne@DravidianLangTech-EACL2021: Transformers with the Class Balanced Loss for Offensive Language Identification in Dravidian Code-Mixed text.