TL;DR
CARE is a self-supervised framework designed to predict the intentions of road users (like pedestrians and cyclists) while remaining robust to changing environmental conditions like weather or time of day. By combining temporally asymmetric contrastive learning with an Invariant Risk Minimization (IRM)-style penalty and auxiliary reconstruction tasks, CARE learns motion representations that generalize better to unseen environments and produce more reliable, well-calibrated probability estimates.
What Problem Does This Paper Address?
Predicting what a pedestrian or cyclist will do next—Behavioural Intention Prediction (BIP)—is a cornerstone of safe autonomous driving. However, trajectory data is "noisy" in a specific way: motion patterns can look different depending on the environment (e.g., people move differently in heavy rain vs. clear sun, or at night vs. noon).
Standard machine learning models often take "shortcuts," latching onto environment-specific regularities that don't hold up when the car moves to a new city or encounters different weather. The authors tackle the challenge of learning environment-invariant representations—features that capture the essence of the intent regardless of the "nuisance" factors of the surroundings.
Methods
The authors introduce CARE, which stands for Contrastive Asymmetric Representation learning with Environment-invariance. The framework (visualized in Figure 1) uses three main pillars:
- Temporally Asymmetric Contrastive Learning: For every trajectory, the model creates three "views": an Anchor (the original), a Positive (where the later part of the window is partially masked), and a Negative (where the later part is corrupted by rotating or reversing it). This forces the model to distinguish between plausible and implausible future motions based on the same early context.
- IRM-style Regularization: To prevent the model from relying on environment-specific cues, they apply a gradient penalty inspired by Invariant Risk Minimization (IRM). This penalizes the model if the "optimal" way to solve the contrastive task varies significantly between different environment groups (e.g., raining vs. sunny).
- Auxiliary Regularization: The model also performs Masked Reconstruction (trying to rebuild the masked parts of the trajectory) and Evidence Consistency. The latter uses evidential learning to ensure the model's internal "uncertainty" parameters remain consistent even when parts of the input are missing.
Datasets
The study uses a dataset derived from the IMPTC (In-vehicle Multi-sensor Pedestrian Trajectory Collection).
- Task: A four-class classification problem: straight, turn left, turn right, and stop.
- Labels: Derived using kinematic rules (e.g., speed thresholds and heading changes).
- Environments: Defined by the joint condition of precipitation intensity (3 levels) and part of day (4 intervals), creating up to 12 distinct environment categories.
Key Insights and Ideas
- Reconstruction is a Powerhouse: In ablation studies, adding the reconstruction task (REC) provided the single largest boost to accuracy and probability quality. It helps the encoder retain fine-grained temporal details that contrastive learning might otherwise discard.
- Calibration over Raw Accuracy: While the IRM penalty didn't always lead to the highest raw accuracy, it was a key driver in reducing Expected Calibration Error (ECE). In safety-critical fields like autonomous driving, knowing how sure a model is (calibration) is often as important as being right.
- Interleaved Embeddings: Qualitative UMAP visualizations (Figures 2-4 in the paper) show that trajectories from different environments are interleaved in the latent space. This suggests the model is successfully avoiding clustering data simply by "Environment ID," focusing instead on the underlying motion geometry.
- The "Invariance" Caveat: The authors are careful to note that using an IRM-style penalty is a regularization strategy, not a formal proof that the model has discovered "causal" factors. It's a practical tool for better generalization, not a silver bullet for perfect robustness.
Key Results
CARE was evaluated against several strong time-series baselines (like DLinear, PatchTST, and TCN) under two protocols: IID (training and testing on the same environment distribution) and Leave-One-Environment-Out (testing on a completely unseen environment).
- IID Performance: CARE achieved 0.840 accuracy and a very low 0.023 ECE, outperforming all baselines in both predictive power and calibration.
- Robustness (Held-out Environment): When tested on an unseen environment, CARE maintained a high 0.806 accuracy.
- Linear Probing: Even when the encoder was "frozen" (linear probing), the full CARE objective yielded the lowest ECE (0.049) among all tested variants, proving the representation itself is inherently more reliable.
Limitations and Future Work
- Proxy Labels: The "intentions" are derived from rules (kinematics) rather than ground-truth human mental states. Future work could involve more complex, human-annotated datasets.
- Long-Tailed Environments: The dataset's environment distribution is highly imbalanced (some weather/time combos are rare). The authors suggest that future evaluations should look at more adequately populated held-out groups.
- Seed Variability: The current results are reported for a single random seed; quantifying the variance across multiple runs would strengthen the reliability of the findings.
- Causal Discovery: While CARE is "causal-inspired," it doesn't explicitly model a causal graph. Integrating formal causal discovery could be a promising next step.
Summary updated