RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting

Jiachen Li

TL;DR

This paper introduces RAIN, a novel framework for motion forecasting that dynamically selects and ranks important historical information using a hybrid attention mechanism. By combining reinforcement learning-based hard attention for selection and soft attention for ranking, RAIN achieves state-of-the-art performance in predicting future trajectories for both multi-agent systems and human skeletons, while also offering interpretable attention weights.

What Problem Does This Paper Address?

Predicting future motion sequences is crucial for applications like autonomous driving and human-robot interaction. However, historical observations often contain irrelevant or distracting information, making it challenging for models to focus on what truly matters for accurate forecasting. Existing methods often struggle to dynamically adapt their attention to varying levels of importance of observed elements, especially in complex, interacting scenarios. This paper aims to solve this by developing a framework that can intelligently discern and prioritize key information from a rich set of observations.

Methods

RAIN employs a hybrid attention mechanism to address the problem of dynamic key information selection and ranking. This mechanism has two core components:

  • Reinforcement Learning (RL)-based Hard Attention: This module acts as a key information selector. It uses RL to learn a policy that decides which pieces of information (represented as edges in a graph for multi-agent systems, or motion segments for human motion) are most relevant. Unlike traditional hard attention, it doesn't enforce a fixed number of selected elements, allowing for flexible selection. This is trained using a reward function that penalizes prediction errors and rewards improvements.
  • Soft Attention: Once the hard attention module has selected the key information, soft attention is used to further rank the relative importance of these selected elements. This allows for a finer-grained understanding of significance.

The framework is designed to be generic and is instantiated for two main tasks:

  • Multi-Agent Trajectory Prediction: Here, agents are represented as nodes in a graph. The RL-HA module learns to select relevant "edges" (interactions) between agents.
  • Human Motion Forecasting: In this case, the RL-HA module selects important "segments" from the historical motion sequence.

A double-stage training pipeline with an alternating strategy is proposed to handle the non-differentiable nature of the RL-HA module. This involves pre-training modules like Graph Message Passing (GMP) and the Soft Graph Attention based Motion Generator (SGA-MG) independently, followed by alternating optimization of RL-HA and SGA-MG in the formal-training stage.

Datasets

The paper evaluates RAIN on several datasets:

  • Synthetic Simulation (Mixed Particle System): A simulated physical system with charged and uncharged particles, used to evaluate relation recognition and trajectory prediction.
  • nuScenes Dataset: A large-scale driving dataset used for multi-agent trajectory prediction of vehicles and pedestrians.
  • Human3.6M Dataset: A widely used dataset for human pose estimation and motion forecasting, used to evaluate human skeleton motion prediction.

Key Insights and Ideas

  • Dynamic Key Information Selection is Crucial: The paper strongly emphasizes that not all historical data is equally important for forecasting. RAIN's ability to dynamically select and rank information is a key differentiator.
  • Hybrid Attention for Complementary Strengths: Combining RL-based hard attention (for selection) with soft attention (for ranking) leverages the strengths of both. Hard attention forces focus, while soft attention refines it.
  • Interpretable Attention Weights: The framework not only achieves high accuracy but also provides attention weights that are interpretable, showing which agents or motion segments were deemed most important. This is a significant advantage for understanding model behavior.
  • Flexibility in Selection: The RL-based hard attention's ability to select an arbitrary number of elements (including none or all) makes it adaptable to varying situational complexities.
  • Effective Training Strategy: The proposed double-stage training pipeline with alternating optimization is essential for effectively training the hybrid attention mechanism.

Key Results

  • State-of-the-Art Performance: RAIN consistently achieves state-of-the-art results across various benchmarks for both multi-agent trajectory prediction (nuScenes) and human motion forecasting (Human3.6M).
  • Improved Prediction Accuracy: For the nuScenes dataset, RAIN (hybrid, dynamic) achieved the smallest error and lowest miss rate for both vehicle and pedestrian prediction. For Human3.6M, it outperformed the previous state-of-the-art method HisRepItself in average accuracy.
  • Enhanced Relation Recognition: In the mixed particle simulation, RAIN achieved the highest recognition accuracy among methods not requiring true relation labels, improving significantly over baselines like DNRI.
  • Interpretable Visualizations: Figures 4 and 7 demonstrate how RAIN's attention maps highlight relevant agents or motion segments, providing visual evidence of its interpretability. In some cases, RAIN correctly identified scenarios where no external agents were influential or where specific historical segments were irrelevant.

Limitations and Future Work

  • RL Training Stability: While the paper proposes an effective training strategy, RL training can still be sensitive to hyperparameters and may require careful tuning. The paper mentions that end-to-end training with ELBO tends to be more stable than RL in general, though it didn't achieve better final performance.
  • "Directions" Action Anomaly: In human motion forecasting, RAIN showed a slight dip in performance on the "Directions" action, potentially due to the lack of clear temporal patterns making it hard for RL to discriminate key segments. This suggests that for highly unpredictable or novel actions, the RL-HA might struggle.
  • Future Work: The paper implicitly suggests that further exploration of RL reward design and more sophisticated attention mechanisms could lead to even greater improvements. The generic nature of RAIN also opens doors for application to other domains requiring dynamic information selection.

Summary updated

Abstract

Motion forecasting plays a significant role in various domains (e.g., autonomous driving, human-robot interaction), which aims to predict future motion sequences given a set of historical observations. However, the observed elements may be of different levels of importance. Some information may be irrelevant or even distracting to the forecasting in certain situations. To address this issue, we propose a generic motion forecasting framework (named RAIN) with dynamic key information selection and ranking based on a hybrid attention mechanism. The general framework is instantiated to handle multi-agent trajectory prediction and human motion forecasting tasks, respectively. In the former task, the model learns to recognize the relations between agents with a graph representation and to determine their relative significance. In the latter task, the model learns to capture the temporal proximity and dependency in longterm human motions. We also propose an effective doublestage training pipeline with an alternating training strategy to optimize the parameters in different modules of the framework. We validate the framework on both synthetic simulations and motion forecasting benchmarks in different domains, demonstrating that our method not only achieves state-of-the-art forecasting performance, but also provides interpretable and reasonable hybrid attention weights.
BibTeX
@inproceedings{li2021rain,
  author = {Jiachen Li},
  title = {{RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=q58y6vPd3gS},
}
Figure