Introduction
Subjective Time-series Regression (STR) involves predicting human-perceived values over time, such as the "excitement" level of a video or the "happiness" of a musical track. Unlike objective tasks, STR labels are inherently noisy and dependent on human cognition. A significant bottleneck in developing robust STR models is the scarcity of annotators; datasets often rely on just a few individuals, leading to a biased "undersampling" of the true population opinion.
The Challenge of Subjective Labels
The core issue in STR is "label distribution bias." When only a few annotators provide scores, the resulting average labels often fail to represent the broader population's consensus. If an infinite number of people were to rate the same data (the "utopia" scenario), the distribution of labels would likely follow a different, smoother pattern.
Standard techniques for addressing imbalanced regression often assume a uniform target distribution and use data augmentation methods that disrupt temporal flow. In time-series data, however, the context provided by neighboring frames or samples is crucial. Randomly duplicating or interpolating data points can destroy this "arrow of time," hindering the model's ability to learn temporal dependencies.
Utopia Label Distribution Approximation (ULDA)
To address these challenges, the authors introduce the Utopia Label Distribution Approximation (ULDA) framework. ULDA aims to transform the biased training distribution into an approximated "utopia" distribution through three key components:
1. Label Distribution Convolution
Invoking the Central Limit Theorem, the authors argue that the collective votes of a large population should approximate a Gaussian distribution. They apply a Gaussian kernel to the existing training labels, smoothing the empirical distribution toward this theoretical ideal. This provides a statistically grounded target for model training.
2. Time-slice Normal Sampling (TNS)
When the utopia distribution requires more samples for certain label values, ULDA employs Time-slice Normal Sampling (TNS). Unlike standard oversampling, TNS respects temporal integrity. It identifies "time-slices" with similar labels, estimates the local feature distribution, and uses Monte Carlo sampling to generate synthetic features that fit seamlessly into the existing timeline.
3. Convolutional Weighted Loss (CWL)
In cases where the utopia distribution requires fewer samples than are present in the dataset, ULDA avoids deleting data—which would create temporal gaps. Instead, it uses Convolutional Weighted Loss (CWL) to down-weight over-represented samples during training, effectively correcting the bias while maintaining the full sequence.
Experimental Results
The ULDA framework was evaluated across several benchmarks, including LIRIS-ACCEDE (movie emotion), SumMe/TVSum (video summarization), and PMEmo (music emotion).
- State-of-the-Art Performance: On the LIRIS-ACCEDE dataset, combining ULDA with a Relative Multi-head Attention Network (RMN) achieved a Pearson Correlation Coefficient (PCC) of 0.531 for Valence and 0.492 for Arousal, setting a new benchmark.
- Video Summarization: On the SumMe dataset, ULDA improved the F1-score of the RMN baseline from 64.0 to 65.8.
- Ablation Studies: Results consistently showed that the combination of TNS and CWL is superior to using either technique in isolation, confirming the necessity of both filling distribution gaps and weighting excesses.
Conclusion
ULDA offers a principled approach to handling the inherent bias in subjective time-series datasets. By grounding label smoothing in statistical theory and developing sampling techniques that respect temporal context, it enables the training of more accurate and fairer regression models. Future work will focus on making TNS more model-agnostic to support a wider range of architectures, including online learners.
Summary updated