CARE: Contrastive Representation Learning with Environment-Invariance Regularization for Behavioural Intention Prediction

Shang Gao, Zhixin Huang, Franz Götz-Hahn, Konrad Doll, Bernhard SickarXiv 2026 · arXiv

TL;DR

CARE is a self-supervised framework designed to predict the intentions of road users (like pedestrians and cyclists) while remaining robust to changing environmental conditions like weather or time of day. By combining temporally asymmetric contrastive learning with an Invariant Risk Minimization (IRM)-style penalty and auxiliary reconstruction tasks, CARE learns motion representations that generalize better to unseen environments and produce more reliable, well-calibrated probability estimates.

What Problem Does This Paper Address?

Predicting what a pedestrian or cyclist will do next—Behavioural Intention Prediction (BIP)—is a cornerstone of safe autonomous driving. However, trajectory data is "noisy" in a specific way: motion patterns can look different depending on the environment (e.g., people move differently in heavy rain vs. clear sun, or at night vs. noon).

Standard machine learning models often take "shortcuts," latching onto environment-specific regularities that don't hold up when the car moves to a new city or encounters different weather. The authors tackle the challenge of learning environment-invariant representations—features that capture the essence of the intent regardless of the "nuisance" factors of the surroundings.

Methods

The authors introduce CARE, which stands for Contrastive Asymmetric Representation learning with Environment-invariance. The framework (visualized in Figure 1) uses three main pillars:

  1. Temporally Asymmetric Contrastive Learning: For every trajectory, the model creates three "views": an Anchor (the original), a Positive (where the later part of the window is partially masked), and a Negative (where the later part is corrupted by rotating or reversing it). This forces the model to distinguish between plausible and implausible future motions based on the same early context.
  2. IRM-style Regularization: To prevent the model from relying on environment-specific cues, they apply a gradient penalty inspired by Invariant Risk Minimization (IRM). This penalizes the model if the "optimal" way to solve the contrastive task varies significantly between different environment groups (e.g., raining vs. sunny).
  3. Auxiliary Regularization: The model also performs Masked Reconstruction (trying to rebuild the masked parts of the trajectory) and Evidence Consistency. The latter uses evidential learning to ensure the model's internal "uncertainty" parameters remain consistent even when parts of the input are missing.
Figure 1: CARE combines temporally asymmetric contrastive views with an environment-wise IRM- style penalty; reconstruction and evidence consistency provide auxiliary regularization. The architecture and computation graph are retained from the original manuscript.

Datasets

The study uses a dataset derived from the IMPTC (In-vehicle Multi-sensor Pedestrian Trajectory Collection).

  • Task: A four-class classification problem: straight, turn left, turn right, and stop.
  • Labels: Derived using kinematic rules (e.g., speed thresholds and heading changes).
  • Environments: Defined by the joint condition of precipitation intensity (3 levels) and part of day (4 intervals), creating up to 12 distinct environment categories.

Key Insights and Ideas

  • Reconstruction is a Powerhouse: In ablation studies, adding the reconstruction task (REC) provided the single largest boost to accuracy and probability quality. It helps the encoder retain fine-grained temporal details that contrastive learning might otherwise discard.
  • Calibration over Raw Accuracy: While the IRM penalty didn't always lead to the highest raw accuracy, it was a key driver in reducing Expected Calibration Error (ECE). In safety-critical fields like autonomous driving, knowing how sure a model is (calibration) is often as important as being right.
  • Interleaved Embeddings: Qualitative UMAP visualizations (Figures 2-4 in the paper) show that trajectories from different environments are interleaved in the latent space. This suggests the model is successfully avoiding clustering data simply by "Environment ID," focusing instead on the underlying motion geometry.
  • The "Invariance" Caveat: The authors are careful to note that using an IRM-style penalty is a regularization strategy, not a formal proof that the model has discovered "causal" factors. It's a practical tool for better generalization, not a silver bullet for perfect robustness.

Key Results

CARE was evaluated against several strong time-series baselines (like DLinear, PatchTST, and TCN) under two protocols: IID (training and testing on the same environment distribution) and Leave-One-Environment-Out (testing on a completely unseen environment).

  • IID Performance: CARE achieved 0.840 accuracy and a very low 0.023 ECE, outperforming all baselines in both predictive power and calibration.
  • Robustness (Held-out Environment): When tested on an unseen environment, CARE maintained a high 0.806 accuracy.
  • Linear Probing: Even when the encoder was "frozen" (linear probing), the full CARE objective yielded the lowest ECE (0.049) among all tested variants, proving the representation itself is inherently more reliable.

Limitations and Future Work

  • Proxy Labels: The "intentions" are derived from rules (kinematics) rather than ground-truth human mental states. Future work could involve more complex, human-annotated datasets.
  • Long-Tailed Environments: The dataset's environment distribution is highly imbalanced (some weather/time combos are rare). The authors suggest that future evaluations should look at more adequately populated held-out groups.
  • Seed Variability: The current results are reported for a single random seed; quantifying the variance across multiple runs would strengthen the reliability of the findings.
  • Causal Discovery: While CARE is "causal-inspired," it doesn't explicitly model a causal graph. Integrating formal causal discovery could be a promising next step.

Summary updated

Abstract

Trajectory-based behavioural intention prediction faces environmental variation, and representations learned with a pooled contrastive objective may retain environment-specific regularities. We present CARE, a self-supervised trajectory-learning framework that couples temporally asymmetric contrastive views with an invariant-risk-minimisation (IRM)-style gradient penalty on environment-wise contrastive risks. Each input window yields an anchor, a partially masked positive view, and a temporally perturbed negative view. Masked reconstruction and observability-aware evidence consistency complement this objective by promoting temporal information retention under partial input. On an IMPTC-derived four-class intention task, CARE achieves 0.840 accuracy and 0.023 expected calibration error (ECE) under IID fine-tuning, and 0.806 accuracy with a Brier score of 0.308 on the reported held-out environment. Under frozen-encoder linear probing, reconstruction provides the largest individual held-out accuracy gain, whereas the complete objective yields the lowest held-out ECE (0.049) among tested variants. These results support the combined objective as a competitive representation-learning approach under the evaluated shift, while the IRM-style regulariser is not taken as evidence of formal invariance.
BibTeX
@misc{gao2026care,
  author = {Shang Gao and Zhixin Huang and Franz Götz-Hahn and Konrad Doll and Bernhard Sick},
  title = {{CARE: Contrastive Representation Learning with Environment-Invariance Regularization for Behavioural Intention Prediction}},
  year = {2026},
  eprint = {2611.02806},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02806},
}
Figure