Abstract
Understanding human intentions in real-world, untrimmed video streams remains a significant challenge due to the prevalence of background noise, long-tail data distributions, and shifting environmental contexts (concept drift). This paper introduces a novel framework for Intentional Evolutionary Learning that predicts the trajectory of human intent—specifically the current action, the subsequent action, and the transition timing. Central to this approach is the ICCA Loss (Instance Confidence and Class Accuracy), a dynamic loss function designed to calibrate model weights in real-time. Evaluated on THUMOS14 and ActivityNet v1.3, the proposed method achieves state-of-the-art performance in both action recognition and intention prediction.
Introduction
Most existing action prediction models are optimized for "trimmed" video clips, where a single, clearly defined action occurs. However, real-world video is typically "untrimmed," characterized by long sequences of background activity interspersed with brief, meaningful actions. This environment presents three primary hurdles: the Long-Tail Problem, where background noise dominates target actions; Concept Drift, where data patterns shift over time; and the limitation of Static Intent, where models only guess the immediate next action rather than the fluid evolution of behavior.
As shown in Figure 1, the framework must distinguish between observed segments and the evolving transition toward future actions. By treating intention as a trajectory rather than a single point, the model can better anticipate human behavior in complex settings.
Methodology
ICCA Loss (Instance Confidence and Class Accuracy)
To address long-tail distributions and concept drift, the authors developed the ICCA Loss. Unlike standard fixed-weight loss functions, ICCA dynamically adjusts based on two factors:
- Instance-level Confidence: The model's certainty regarding a specific sample in the current stream.
- Class Accuracy: The model's historical performance on that category in the previous training epoch. This self-calibration allows the model to adapt to new data distributions without requiring immediate ground-truth labels for the incoming stream.
Intention-Oriented Evolutionary Learning
The framework processes video features (extracted via I3D) through a recurrent neural network (RNN) to maintain a "best potential intention." The core output is an Evolution Triplet:
- Current Action ($\hat{y}$): The action currently being performed.
- Predicted Next Action ($\hat{y}'$): The anticipated subsequent behavior.
- Remaining Duration ($\tilde{D}$): The estimated time until the transition occurs.
As illustrated in Figure 5, an "Intention Coherence" loss ensures that these predictions remain semantically consistent over time, preventing erratic jumps in predicted intent.
Experimental Results
The researchers evaluated the framework on the THUMOS14 and ActivityNet v1.3 benchmarks.
- THUMOS14: The model achieved 67.9% Top-1 recognition accuracy and 64.8% prediction accuracy, outperforming previous state-of-the-art (SOTA) methods like RULSTM, which typically reached ~60% and ~56% respectively.
- ActivityNet v1.3: Prediction accuracy reached 45.7%, a significant jump from the previous SOTA of 39.9%.
Crucially, the model demonstrated superior precision in timing, accurately estimating when an action would evolve. This temporal awareness is vital for applications such as autonomous safety systems and human-robot collaboration.
Conclusion
Intentional Evolutionary Learning provides a robust solution for interpreting human behavior in messy, real-world video. By leveraging the ICCA Loss to handle data imbalance and modeling intent as a dynamic triplet, the framework moves beyond simple action recognition toward a deeper understanding of human trajectories. Future work will focus on refining the precision of transition timing and optimizing the "fusion frequency" for real-time adaptation.
Summary updated