TL;DR
This paper introduces a framework that jointly optimizes model partitioning and per-layer quantization for Vision Transformers (ViTs) in vehicular edge computing. By formulating the problem as a Mixed-Integer Linear Program (MILP), the authors significantly reduce energy consumption and latency for multi-stream perception tasks while maintaining high 3D detection accuracy.
What Problem Does This Paper Address?
Connected and Automated Vehicles (CAVs) rely on complex Vision Transformer models for situational awareness. However, these models are computationally expensive and memory-intensive, often exceeding the capabilities of local on-vehicle hardware.
While "split inference"—executing the first few layers on the vehicle and offloading the rest to a more powerful edge aggregator—is a known solution, existing methods are often static. They fail to adapt to dynamic conditions like fluctuating V2V (Vehicle-to-Vehicle) bandwidth, varying processing loads, or multiple concurrent sensor streams. Furthermore, most research treats model quantization (reducing numerical precision to save space) and partitioning as separate problems. This paper argues that because quantization directly changes the size of the data being transmitted, these two decisions must be made together to achieve true efficiency.
Methods
The authors propose a joint optimization framework called Joint DynOpt. Its core is a Mixed-Integer Linear Program (MILP) that minimizes the total energy (compute + communication) across all concurrent streams.
Key technical components include:
- Joint Decision Space: The optimizer simultaneously selects the partition point for each stream, the node assignment (local vs. aggregator), and the per-layer precision (INT8 vs. FP16).
- Exact Linearization: To keep the MILP solvable in real-time, the authors use auxiliary binary variables and "McCormick Envelopes" to linearize the coupled relationships between precision, activation size, and partition points.
- Context-Awareness: The model accounts for profiled layer latencies that change based on the number of concurrent streams ($K$) and stream-specific V2V link rates.
- Quality Constraints: Instead of just maximizing accuracy, the framework treats task quality as a constraint, using an empirically calibrated "sensitivity budget" to ensure the final 3D detection performance remains acceptable.
Datasets
The framework was evaluated using the OPV2V benchmark, a large-scale dataset for V2V collaborative perception.
- Scenarios: The evaluation used LiDAR and camera perception traces from CARLA (simulated) and Culver City (real-world) environments.
- Model: A 12-layer Vision Transformer backbone was used as the primary computational workload.
- Hardware Testbed: A heterogeneous setup featuring NVIDIA Jetson Orin Nano and Orin NX as local nodes, and a Jetson AGX Orin as the edge aggregator.
Key Insights and Ideas
- The Quantization-Communication Link: The "aha moment" here is that quantization isn't just about faster math; it's a compression tool for the network. Selecting INT8 for a layer right before a split point can halve the transmission energy for that stream.
- Non-Monotonic Sensitivity: The authors found that quantization sensitivity isn't uniform across a Transformer. Some layers (like 1, 6, 9, 11, and 12) are highly sensitive and should stay in FP16, while others can be aggressively quantized to INT8 without hurting the final mAP (mean Average Precision) significantly.
- Real-Time Feasibility: Despite the complexity of MILP, the solver (SCIP 8.0) consistently found optimal solutions in under 10ms on the Jetson AGX Orin, making it suitable for the high-speed requirements of vehicular networking.
Key Results
The Joint DynOpt framework was compared against several baselines, including full local execution and uniform INT8 splits.
- Energy Savings: The proposed method reduced V2V communication energy by 33.3–36.0% compared to a uniform INT8 split and by up to 70.1% compared to static FP16 splits.
- Latency Reduction: At a high workload of 8 concurrent streams ($K=8$), end-to-end latency dropped by 34.0% on the Orin Nano and 33.1% on the Orin NX relative to the uniform INT8 baseline.
- Accuracy Trade-off: The framework limited the drop in 3D detection accuracy to just 3.1–4.1 percentage points (mAP50) compared to the "gold standard" full FP16 execution, while being significantly faster and more energy-efficient.
Limitations and Future Work
The authors acknowledge a few areas for expansion:
- Precision Granularity: The current work focuses on INT8 and FP16. Future iterations could explore sub-8-bit quantization (e.g., INT4) for even greater savings.
- Semantic Communication: The paper suggests that further gains could be made by incorporating "token reduction" techniques—identifying and discarding less important visual tokens before transmission.
- Dynamic Link Conditions: While the model handles different link rates, testing under more extreme mobility-induced fading or interference would further validate its robustness in real-world V2X scenarios.
Summary updated