A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning

Liting Sun, Cheng Peng, Wei Zhan, Masayoshi TomizukaOpenReview 2017 · DOI · OpenReview

Abstract

For safe and efficient planning and control in autonomous driving, we need a driving policy which can achieve desirable driving quality in long-term horizon with guaranteed safety and feasibility. Optimization-based approaches, such as Model Predictive Control (MPC), can provide such optimal policies, but their computational complexity is generally unacceptable for real-time implementation. To address this problem, we propose a fast integrated planning and control framework that combines learning- and optimization-based approaches in a two-layer hierarchical structure. The first layer, defined as the "policy layer", is established by a neural network which learns the long-term optimal driving policy generated by MPC. The second layer, called the "execution layer", is a short-term optimization-based controller that tracks the reference trajectories given by the "policy layer" with guaranteed short-term safety and feasibility. Moreover, with efficient and highly-representative features, a small-size neural network is sufficient in the "policy layer" to handle many complicated driving scenarios. This renders online imitation learning with Dataset Aggregation (DAgger) so that the performance of the "policy layer" can be improved rapidly and continuously online. Several exampled driving scenarios are demonstrated to verify the effectiveness and efficiency of the proposed framework.

Introduction

Autonomous driving systems must simultaneously satisfy three often-conflicting objectives: safety/feasibility, optimality (driving comfort and efficiency), and real-time responsiveness. While traditional Model Predictive Control (MPC) excels at satisfying safety constraints and optimizing long-term trajectories, its computational burden often precludes real-time application in complex, dynamic environments. Conversely, end-to-end deep learning approaches offer rapid inference but often lack formal safety guarantees and require immense computational resources for training. This paper introduces a hierarchical framework that "distills" the foresight of a long-term MPC expert into a fast neural network policy, coupled with a short-term MPC for immediate safety enforcement.

Methodology

The proposed framework utilizes a two-layer hierarchical structure to bridge the gap between long-term planning and real-time control.

The proposed hierarchical integrated planning and control framework.
  1. Policy Layer (Neural Network): This layer acts as the "brain," learning to mimic the behavior of a computationally expensive, long-term MPC expert (e.g., a 30-step horizon). Instead of raw sensor data, it processes abstracted "middle-level" features. Crucially, it only predicts the "proximate" or immediate portion of the long-term optimal trajectory, simplifying the learning task while retaining the expert's strategic intent.
  2. Execution Layer (Short-term MPC): This layer serves as the "reflex," taking the reference trajectory from the policy layer and generating final control actions (acceleration $a$ and steering $\delta$). By using a very short horizon (e.g., 5 steps), it remains computationally light while providing a mathematical guarantee of safety and feasibility.

To train the policy layer, the authors employ Sampled-DAgger, a variant of Dataset Aggregation. This active learning approach allows the system to drive using its current policy while an expert MPC runs in parallel. Scenarios where the policy significantly deviates from the expert are added to the training set, making the model robust to its own potential errors.

Feature Engineering and Learning

A key innovation is the use of high-level feature engineering rather than raw pixel data. By abstracting the environment into distances to curbs, lane deviations, and relative velocities of surrounding vehicles, the researchers significantly reduced the complexity of the neural network.

Illustration of the middle-level features extracted for the policy layer.

This abstraction allows for a small, single-hidden-layer network that can be trained in minutes on a standard CPU. Furthermore, complex multi-lane scenarios are generalized by treating lane markers as "virtual curbs" and surrounding vehicles as dynamic obstacles, allowing the same policy to handle diverse driving maneuvers like overtaking.

Results

The framework demonstrates exceptional computational efficiency. The policy layer executes in approximately 6 microseconds, while the execution layer requires roughly 76 milliseconds. The total runtime is well within the 100ms threshold required for 10Hz real-time control.

In simulation, the integrated framework successfully performed complex maneuvers such as smooth overtaking. In contrast, a standalone short-horizon MPC (lacking the policy layer's long-term guidance) failed these tasks, resulting in late braking or collisions with curbs. The Sampled-DAgger process was also shown to effectively correct initial model deficiencies, such as a failure to maintain a straight path, after only a few iterations of online data collection.

Conclusion

This research presents a viable path for integrating the strengths of optimization-based control and imitation learning. By delegating long-term strategic planning to a fast neural network and short-term safety to a lightweight MPC, the framework achieves real-time performance with high driving quality. Future work will focus on extending this approach to more complex urban environments, including intersections and curved roads, and addressing potential uncertainties in the perception module's output.

Summary updated

Abstract

For safe and efficient planning and control in autonomous driving, we need a driving policy which can achieve desirable driving quality in long-term horizon with guaranteed safety and feasibility. Optimization-based approaches, such as Model Predictive Control (MPC), can provide such optimal policies, but their computational complexity is generally unacceptable for real-time implementation. To address this problem, we propose a fast integrated planning and control framework that combines learning- and optimization-based approaches in a two-layer hierarchical structure. The first layer, defined as the "policy layer", is established by a neural network which learns the long-term optimal driving policy generated by MPC. The second layer, called the "execution layer", is a short-term optimization-based controller that tracks the reference trajecotries given by the "policy layer" with guaranteed short-term safety and feasibility. Moreover, with efficient and highly-representative features, a small-size neural network is sufficient in the "policy layer" to handle many complicated driving scenarios. This renders online imitation learning with Dataset Aggregation (DAgger) so that the performance of the "policy layer" can be improved rapidly and continuously online. Several exampled driving scenarios are demonstrated to verify the effectiveness and efficiency of the proposed framework.
BibTeX
@misc{sun2017fast,
  author = {Liting Sun and Cheng Peng and Wei Zhan and Masayoshi Tomizuka},
  title = {{A Fast Integrated Planning and Control Framework for Autonomous Driving via Imitation Learning}},
  year = {2017},
  eprint = {1707.02515},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1707.02515},
  howpublished = {OpenReview},
}
Figure