Abstract
MT-Net is a high-performance 3D object detection framework designed to enhance the popular CenterPoint architecture for autonomous driving applications. By integrating an optimal transport-based target assignment strategy (OTA) and a multi-scale 2D backbone, MT-Net addresses common challenges such as fixed supervision and scale variance in Lidar-only point clouds. The framework achieved a competitive score of 78.45 mAPH, securing 4th place on the Waymo 3D Detection leaderboard.
Introduction
Lidar-only 3D object detection is a cornerstone of autonomous driving perception. While architectures like CenterPoint have established strong performance baselines, they are often limited by two primary factors: fixed supervision and scale variance. Standard models typically assign ground-truth labels only to the exact center of an object, neglecting informative surrounding points. Furthermore, a single-scale feature map often fails to capture the diverse range of object sizes—from small pedestrians to large semi-trucks—present in driving environments. MT-Net, developed by researchers at Meituan, introduces several architectural and training innovations to overcome these hurdles.
Methods
The MT-Net pipeline follows a "Voxel-to-3D-to-2D" architecture, as illustrated in Figure 1.
The framework introduces several key technical components:
- Optimal Transport Assignment (OTA): Instead of rigid center-only assignment, MT-Net treats label assignment as an optimal transport problem. It uses a dynamic strategy to calculate a cost matrix based on classification and regression losses, allowing the model to negotiate which predicted candidates best represent ground-truth objects.
- Dual 2D Backbones: To handle scale variance, MT-Net employs two distinct 2D backbones after collapsing 3D features into a pseudo-image. 2D-A utilizes self-calibrated convolutions (SC-Conv) for multi-scale feature combination, while 2D-B implements a Feature Pyramid Network (FPN) structure for improved detection across varying object sizes.
- GT-Paste Fading: Virtual objects are "pasted" into training frames to increase data diversity. To prevent overfitting to these synthetic distributions, this augmentation is disabled during the final five epochs of training.
- Inference Strategies: The system employs Test-Time Augmentation (TTA) and Weighted Box Fusion (WBF) to ensemble results from multiple views and models, providing a more robust final output than standard Non-Maximum Suppression (NMS).
Experimental Results
MT-Net was evaluated on the Waymo Open Dataset (WOD), which consists of 230k frames and approximately 12 million annotated 3D boxes. The effectiveness of each component was validated through an extensive ablation study.
As shown in Figure 3, the baseline CenterPoint++ model achieved 71.31 mAPH. The addition of Optimal Transport Assignment and TTA/WBF raised performance to 74.98 mAPH. The most significant single improvement came from the FPN-based 2D backbone, which boosted the score by 2.45 mAPH to 78.61. The final model, incorporating GT-Paste Fading, reached 80.13 mAPH on the validation set and 78.45 mAPH on the official leaderboard.
Conclusion
MT-Net demonstrates that dynamic label assignment and multi-scale feature extraction are critical for high-accuracy 3D object detection. While the framework achieves state-of-the-art results on the Waymo leaderboard, its reliance on model ensembling and TTA introduces significant computational overhead, making it more suitable for offline processing or competitions than real-time on-vehicle deployment. Future work may involve the integration of camera data to further improve detection in sparse Lidar regions.
Summary updated