In the quest for safe autonomous driving, predicting the future trajectories of surrounding agents is a fundamental challenge. For years, the dominant paradigm was "agent-centric": to predict where a specific car would go, the model would shift the entire world's coordinate system so that car was at the origin $(0,0)$. While this simplifies the geometry for a single agent, it creates a massive computational bottleneck when a vehicle needs to track fifty agents at a busy intersection.
Recent research has converged on a more scalable alternative: Scene-Centric Modeling. By processing the environment once and using clever architectural symmetries, these models can predict the futures of every agent in a scene simultaneously without redundant computation.
The Agent Centric Bottleneck
Traditional models suffer from what Jiquan Ngiam and colleagues describe as a redundancy problem. In an agent-centric view, if there are $N$ agents, the high-definition map and the relative positions of all other actors must be re-normalized and re-encoded $N$ times. As Zhejun Zhang notes in the work on HPTR, this leads to poor scalability; the computational cost grows linearly with the number of agents, making real-time deployment difficult in dense urban environments.
This inefficiency isn't just a matter of speed. Juanwu Lu points out that this approach hinders the ability of autonomous systems to perform "what-if" reasoning, as every hypothetical change in the scene would require a full re-processing of the environmental context.
Unified Scene Encoding
The shift toward scene-centricity began with the Scene Transformer. Instead of centering on one agent, Jiquan Ngiam proposed a unified architecture that treats all agents and map elements as tokens in a single Transformer. By using a masking strategy similar to BERT, the model learns the "joint" future of the entire scene. This allows the system to process the map once and predict trajectories for all participants in a single forward pass.
Building on this, Li Jiang and Shaoshuai Shi introduced MTR++, which utilizes Symmetric Scene Modeling. Rather than relying on a global coordinate frame that might be sensitive to rotation, MTR++ uses learnable "intention queries." These queries act as anchors that look at the global scene features and refine them into specific agent paths, maintaining high accuracy while remaining computationally lean.
Hierarchical Invariance
A key technical hurdle in scene-centric models is maintaining "viewpoint invariance"—the idea that a car's behavior shouldn't change just because we've shifted the map's origin.
Zikang Zhou solved this in HiVT using a Hierarchical Vector Transformer. The intuition is to process information at two levels:
- Local: Each agent extracts features from its immediate surroundings in its own local frame.
- Global: These local features are then passed to a global transformer that models the interactions between agents using translation-invariant relative spatial encodings.
This "local-to-global" strategy, also echoed in the TSGN framework, ensures that the model benefits from the efficiency of a single scene-wide pass while retaining the geometric precision of agent-centric views.
From Agents to Instances
The most recent evolution in this direction is the move toward Instance-Centric representations. Fabian Konstantinidis and Moritz Sackmann argue that by treating every car and map element as an independent "instance" with its own local coordinate frame, models can achieve massive data reuse.
In their recent work, they demonstrated that this approach enables a 13.2x speedup in inference. By decoupling the features from a fixed global grid, the model can "freeze" computation for agents with stable trajectories—a concept Wenjian Sun calls Trajectory Entropy—and focus its processing power only on the most uncertain or interactive parts of the scene.
Why It Works
The intuition behind the success of scene-centricity is shared context. In any given traffic scene, the "rules of the road" (the map) and the "state of the world" (the positions of others) are common knowledge to all participants.
By moving away from agent-centric re-normalization, researchers like Hao Cheng with GATraj have shown that we can achieve inference speeds of 100 Hz. The model no longer wastes time re-learning that a stop sign is a stop sign for every car that approaches it; instead, it learns the scene once and applies that understanding to everyone within it.