A Lightweight Target-Driven Network of Stereo Matching for Inland Waterways

Jing Su, Yiqing Zhou, Yu Zhang, Chao Wang, Yi Wei

Abstract

Stereo matching for inland waterways is one of the key technologies for the autonomous navigation of Unmanned Surface Vehicles (USVs), which involves dividing the stereo images into reference images and target images for pixel-level matching. However, due to the challenges of the inland waterway environment, such as blurred textures, large spatial scales, and computational resource constraints of the USVs platform, the participation of geometric features from the target image is required for efficient target-driven matching. Based on this target-driven concept, we propose a lightweight target-driven stereo matching neural network, named LTNet. Specifically, a lightweight and efficient 4D cost volume, named the Geometry Target Volume (GTV), is designed to fully utilize the geometric information of target features by employing the shifted target features as the filtered feature volume. Subsequently, to address the substantial texture interference and object occlusions present in the waterway environment, a Left-Right Consistency Refinement (LRR) module is proposed. The LRR utilizes the pixel-level differences in left and right disparities to introduce soft constraints, thereby enhancing the accuracy of predictions during the intermediate stages of the network. Moreover, knowledge distillation is utilized to enhance the generalization capability of lightweight models on the USVInland dataset. Furthermore, a new large-scale benchmark, named Spring, is utilized to validate the applicability of LTNet across various scenarios. In experiments on the aforementioned two datasets, LTNet achieves competitive results, with only 3.7M parameters.

Introduction

Autonomous navigation for Unmanned Surface Vehicles (USVs) in inland waterways presents unique challenges compared to urban autonomous driving. Waterway environments are often "unstructured," featuring blurred textures of moving water, chaotic vegetation, and large spatial scales where objects are far away. Traditional high-accuracy stereo matching models are often too computationally expensive for edge deployment on USVs, while existing lightweight models often struggle with the lack of distinct geometric features in natural river scenes.

Proposed Method: LTNet

To address these challenges, we propose LTNet, a lightweight target-driven stereo matching network. LTNet balances efficiency and accuracy through three primary innovations:

1. Geometry Target Volume (GTV)

Standard 4D cost volumes are often either computationally heavy or too simplified to capture complex scene geometry. LTNet introduces the Geometry Target Volume (GTV), which explicitly utilizes geometric features from the target image to filter the cost volume. By employing shifted target features as a filtered feature volume, the network can better reconstruct the geometric structure of the scene even when textures are sparse or blurry.

2. Left-Right Consistency Refinement (LRR)

To handle texture interference and object occlusions, we propose the Left-Right Consistency Refinement (LRR) module. This module predicts both left and right disparities at an intermediate stage and calculates their pixel-level differences. These differences serve as a "soft attention" mask, guiding the network to focus its refinement efforts on areas with high inconsistency before final upsampling.

The architecture of LTNet, featuring the Geometry Target Volume (GTV) and Left-Right Consistency Refinement (LRR).

3. Knowledge Distillation

To enhance generalization on real-world datasets like USVInland, which often have sparse LiDAR ground truth, we employ a two-stage knowledge distillation process. A large-scale "Teacher" network (CroCo-Stereo) provides dense, high-quality labels to guide the training of the "Student" LTNet, helping it learn robust features that might be missed by sparse supervision alone.

Experimental Results

LTNet was evaluated on the Scene Flow, USVInland, and Spring datasets. With only 3.7M parameters, LTNet achieves a highly competitive balance between size and performance.

On the USVInland dataset, LTNet achieved a 15.66% 1-pixel error rate, which further improved to 15.36% with knowledge distillation. This performance represents a significant improvement over other lightweight competitors such as CGI-Stereo and MSNet3D, while maintaining a significantly lower parameter count than large-scale models like IGEV-Stereo (12.6M) or the teacher model CroCo-Stereo (437.4M).

Comparison of USVInland error vs. model parameters, highlighting LTNet's efficiency and accuracy relative to other state-of-the-art methods.

Furthermore, on the high-detail Spring benchmark, LTNet ranked 1st in outlier rate (D1) and absolute error (Abs) metrics among compared methods, demonstrating its robustness across diverse scenarios beyond waterways.

Conclusion

LTNet provides an efficient solution for stereo matching on resource-constrained USV platforms. By focusing on target-driven geometry and self-correcting refinement, it achieves state-of-the-art results in challenging inland waterway environments. Future work will focus on addressing the complexities of reflective and transparent water surfaces and improving temporal consistency for video-based navigation.

Summary updated

Abstract

Stereo matching for inland waterways is one of the key technologies for the autonomous navigation of Unmanned Surface Vehicles (USVs), which involves dividing the stereo images into reference images and target images for pixel-level matching. However, due to the challenges of the inland waterway environment, such as blurred textures, large spatial scales, and computational resource constraints of the USVs platform, the participation of geometric features from the target image is required for efficient target-driven matching. Based on this target-driven concept, we propose a lightweight target-driven stereo matching neural network, named LTNet. Specifically, a lightweight and efficient 4D cost volume, named the Geometry Target Volume (GTV), is designed to fully utilize the geometric information of target features by employing the shifted target features as the filtered feature volume. Subsequently, to address the substantial texture interference and object occlusions present in the waterway environment, a Left-Right Consistency Refinement (LRR) module is proposed. The \text{LRR} utilizes the pixel-level differences in left and right disparities to introduce soft constraints, thereby enhancing the accuracy of predictions during the intermediate stages of the network. Moreover, knowledge distillation is utilized to enhance the generalization capability of lightweight models on the USVInland dataset. Furthermore, a new large-scale benchmark, named Spring, is utilized to validate the applicability of LTNet across various scenarios. In experiments on the aforementioned two datasets, LTNet achieves competitive results, with only 3.7M parameters. The code is available at https://github.com/Open-YiQingZhou/LTNet .
BibTeX
@misc{su2024lightweight,
  author = {Jing Su and Yiqing Zhou and Yu Zhang and Chao Wang and Yi Wei},
  title = {{A Lightweight Target-Driven Network of Stereo Matching for Inland Waterways}},
  year = {2024},
  eprint = {2410.07915},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.07915},
}
Figure