VaPiD: A Rapid Vanishing Point Detector via Learned Optimizers

Shichen Liu, Yichao Zhou, Yajie ZhaoICCV 2021 · DOI

Introduction

Vanishing points (VPs) are critical geometric primitives in computer vision, representing the projection of parallel lines in 3D space onto a 2D image plane. They serve as fundamental anchors for camera calibration, autonomous navigation, and 3D scene reconstruction. Traditionally, VP detection relied on a two-step pipeline: extracting line segments followed by clustering. While effective in structured environments, these methods are often computationally intensive and sensitive to noise.

Recent deep learning approaches, such as NeurVPS, have introduced "conic convolutions" to directly learn line structures pointing toward candidate VPs. While robust, these methods suffer from high latency—often limited to 1 FPS—due to the exhaustive search required over a dense grid of potential VP candidates. VaPiD addresses this efficiency gap by introducing a "propose-and-refine" strategy that achieves real-time performance without sacrificing precision.

The VaPiD Architecture

VaPiD employs a two-stage deep learning framework designed to mimic human visual perception: identifying a coarse region of interest and then iteratively refining the estimate for high precision.

The VaPiD architecture consists of a backbone, a Vanishing Point Proposal Network (VPPN), and a Recurrent Vanishing Point Optimizer (NVPO).

1. Vanishing Point Proposal Network (VPPN)

The first stage generates coarse VP candidates using Efficient Conic Convolutions. Standard conic convolutions are expensive because they require unique kernels for every possible VP location. VaPiD optimizes this by pre-computing a discrete set of rotated feature maps. By approximating the conic convolution through a simple lookup of the nearest pre-computed rotation, the computational overhead is drastically reduced, enabling rapid proposal generation.

2. Neural Vanishing Point Optimizer (NVPO)

Rather than performing a fine-grained grid search, VaPiD treats refinement as a learned optimization problem. The NVPO is a recurrent module that takes the coarse proposals and iteratively predicts a residual "nudge" to move the candidate closer to the true vanishing point. This process occurs in a local spherical coordinate system, which prevents mathematical singularities and ensures consistent accuracy across different camera orientations.

Experimental Results

The model was rigorously evaluated across several datasets, including the synthetic SU3 Wireframe and real-world datasets like HoliCity and NYU-VP.

VaPiD achieves a superior trade-off between speed (FPS) and accuracy (median angle error) compared to NeurVPS and traditional methods.

Speed and Precision

VaPiD demonstrates a significant leap in performance, operating at 26 to 43 FPS on standard hardware. This represents a 17x speedup over previous state-of-the-art deep learning methods. Despite this speed, accuracy remains exceptional; on the SU3 dataset, the model achieved a median angle error of 0.088°, effectively reaching the numerical limits of 32-bit floating-point precision.

Robustness and Generalization

Qualitative analysis indicates that VaPiD is highly robust to environmental challenges such as heavy occlusions, shadows, and complex textures (e.g., clouds) that typically confound traditional line-clustering algorithms. Furthermore, the model generalizes well across diverse indoor and outdoor environments, even when trained on synthetic data.

Conclusion

VaPiD represents a paradigm shift in vanishing point detection, moving from exhaustive search to learned, iterative refinement. By combining efficient feature extraction with a neural optimizer, it enables real-time, high-precision geometric reasoning. Future work will explore the integration of VaPiD into end-to-end SLAM systems and autonomous robotics pipelines, where its speed and accuracy can be fully leveraged for real-time spatial awareness.

Summary updated

BibTeX
@inproceedings{liu2021vapid,
  author = {Shichen Liu and Yichao Zhou and Yajie Zhao},
  title = {{VaPiD: A Rapid Vanishing Point Detector via Learned Optimizers}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.01262},
}
Figure