Deep Learning for Vanishing Point Detection Using an Inverse Gnomonic Projection

Florian Kluger, Hanno Ackermann, Michael Ying Yang, Bodo Rosenhahn

Abstract

Vanishing point (VP) detection is a fundamental task in computer vision for scene understanding and camera calibration. Traditional methods often rely on restrictive geometric assumptions or require extensive manually labeled real-world datasets. This paper presents a novel deep learning framework that maps line segments onto a Gaussian sphere via an inverse gnomonic projection. By training a convolutional neural network (CNN) exclusively on synthetic data, the proposed method achieves state-of-the-art performance on diverse real-world benchmarks without the need for Manhattan world assumptions or human-annotated training images.

Introduction

Vanishing points—the points where parallel lines in 3D space appear to converge in a 2D image—are vital for understanding scene geometry, camera calibration, and 3D reconstruction. However, detecting them is notoriously difficult. Traditional methods often rely on the Manhattan world assumption (assuming all lines are aligned with three mutually orthogonal axes) or the Atlanta world assumption (multiple horizontal directions but a shared vertical zenith). These assumptions fail in many real-world environments. Furthermore, VPs can exist "at infinity" relative to the image plane, making them mathematically awkward to represent in standard pixel coordinates. Finally, deep learning approaches usually require massive amounts of labeled real-world data, which is incredibly tedious to produce for vanishing points.

Proposed Methodology

The authors propose a multi-stage pipeline that sidesteps these issues by transforming the problem from the image plane to a spherical representation.

Overview of the proposed vanishing point detection pipeline, from input image to final result.

The pipeline consists of the following steps:

  1. Line Detection: The Line Segment Detector (LSD) identifies edges in the input image.
  2. Inverse Gnomonic Projection: Instead of feeding raw images into a CNN, line segments are mapped onto a Gaussian sphere. This transformation converts the unbounded image plane into a bounded spherical surface. Parallel lines become great circles, and vanishing points become specific coordinates (azimuth and elevation).
  3. CNN Classification: A modified AlexNet architecture performs multi-label classification over an $N \times N$ grid on the sphere's surface, producing a likelihood map of potential vanishing points.
  4. EM Refinement: To achieve high precision, an Expectation-Maximization (EM) algorithm refines the coarse grid predictions. A novel spatial consistency prior helps the model ignore spurious VPs caused by coincidental line intersections in cluttered areas like foliage.

Synthetic Training and Data

A key innovation of this work is the use of purely synthetic data for training. Because the network operates on abstract line representations rather than natural textures, the authors trained it entirely on 576,000 synthetically generated examples.

These examples consist of random 3D line clusters projected into 2D with added noise and outliers. This approach proves that for geometric tasks, a CNN can learn the underlying logic of converging lines from synthetic wireframes and generalize effectively to real-world imagery.

Experimental Results

The model was evaluated on three real-world benchmarks: the York Urban Dataset (YUD), the Eurasian Cities Dataset (ECD), and the Horizon Lines in the Wild (HLW) dataset. The method achieved an Area Under the Curve (AUC) of 94.27% on YUD and 86.26% on ECD, showing strong performance in both Manhattan and non-Manhattan scenes. On the challenging HLW dataset, it achieved 57.31% AUC, significantly outperforming older geometric methods.

The system demonstrates that representing VPs on a sphere eliminates the "infinity" problem, making them easier for CNNs to process. Furthermore, the model's lack of world assumptions allows it to detect an arbitrary number of VPs (up to 6), providing flexibility for complex architectural or natural scenes.

Conclusion and Future Work

This research demonstrates that deep learning for geometric tasks can be successfully performed using synthetic data and spherical projections. While the method is robust, it faces challenges in highly cluttered scenes or those with large curved structures that mimic converging lines. Future work will focus on optimizing the EM refinement step—which currently accounts for 95% of the 45-second processing time per image—and exploring higher-resolution CNN outputs to reduce post-processing overhead.

Summary updated

BibTeX
@inproceedings{kluger2017deep,
  author = {Florian Kluger and Hanno Ackermann and Michael Ying Yang and Bodo Rosenhahn},
  title = {{Deep Learning for Vanishing Point Detection Using an Inverse Gnomonic Projection}},
  booktitle = {GCPR},
  year = {2017},
  doi = {10.1007/978-3-319-66709-6\_2},
}
Figure