On Localizing a Camera from a Single Image

Pradipta Ghosh, Xiaochen Liu, Hang Qiu, Marcos Augusto M. Vieira, Gaurav S. Sukhatme, Ramesh GovindanarXiv 2020 · arXiv

Abstract

Smart cities rely on millions of surveillance cameras, yet these "cameras in the wild" often lack the precise metadata—location, focal length, and orientation—needed to map visual events to physical coordinates. This paper introduces CamLoc, a system that achieves meter-level localization from a single image by combining projective geometry, human-in-the-loop annotations, and deep learning. Unlike purely neural approaches that struggle with perspective shifts, CamLoc leverages orthogonal vanishing points and reference objects for scale. Evaluations show that CamLoc outperforms state-of-the-art models like PoseNet by two orders of magnitude, positioning 95% of test images within 12 meters.

Introduction

The proliferation of public and surveillance cameras in urban environments has created a vast network of visual sensors. However, without knowing a camera's precise GPS location, focal length, and viewing direction (pose), it is impossible to accurately translate pixels into real-world coordinates. Existing solutions generally fall into two categories: robotics-based approaches that require specialized sensors (like LiDAR) or multiple images, and deep learning models (e.g., PoseNet) that often fail to generalize across different camera heights and perspectives. CamLoc addresses this gap by determining the smallest set of additional information required to localize a camera to within meter-level accuracy from a single static feed.

Methodology

CamLoc solves for 18 unknown parameters, including intrinsic camera properties and extrinsic rotation and translation. The system integrates three core components:

  1. Projective Geometry: The system identifies three orthogonal vanishing points—points where parallel lines in 3D space appear to converge in the 2D image. These points allow for the calculation of the camera's internal geometry and orientation.
  2. Reference Objects for Scale: To determine absolute distance (translation), CamLoc uses a "ruler" in the scene. It employs a Deep Neural Network (SSD) to detect vehicles and utilizes standard "average sedan" dimensions to establish physical scale.
  3. Absolute Localization: Relative coordinates are mapped to GPS locations by matching visible road intersections or landmarks to satellite imagery.
Overview of CamLoc's geometric framework (a, b) and scale estimation using a reference vehicle (c).

Human-in-the-Loop Robustness

A significant challenge in urban localization is the "noise" of complex textures, which often causes automated vanishing point detection to fail. CamLoc addresses this by incorporating human perception via crowd-tasking. Amazon Mechanical Turk (AMT) workers are tasked with identifying parallel lines, such as building edges or car dimensions. This "wisdom of the crowds" provides the clean geometric data necessary for precise mathematical modeling.

The human-in-the-loop interface used by crowd workers to annotate parallel lines and reference objects.

Experimental Results

The system was tested on a diverse dataset of 214 images, including Google Street View, smartphone captures, and public webcams. Key findings include:

  • High Precision: CamLoc positioned 95% of test images within 12 meters of the ground truth. In the same environments, PoseNet's error was typically two orders of magnitude higher.
  • Vertical Accuracy: Camera height was estimated to within 1 meter in 80% of cases.
  • Efficiency: The algorithm processes an image in approximately 0.4 seconds once annotations are provided.
  • Virtual Sensing: Once localized, a standard camera can function as a "virtual sensor," estimating vehicle speeds within a 15% error margin or measuring building heights without specialized hardware.

Conclusion

CamLoc demonstrates that the mathematical rigor of projective geometry remains superior to pure deep learning for spatial reasoning tasks. By using neural networks for object detection and humans for robust feature identification, the system achieves unprecedented localization accuracy from a single image. Future work will focus on automating the annotation process and utilizing temporal data from video feeds to further refine results.

Summary updated

Abstract

Public cameras often have limited metadata describing their attributes. A key missing attribute is the precise location of the camera, using which it is possible to precisely pinpoint the location of events seen in the camera. In this paper, we explore the following question: under what conditions is it possible to estimate the location of a camera from a single image taken by the camera? We show that, using a judicious combination of projective geometry, neural networks, and crowd-sourced annotations from human workers, it is possible to position 95% of the images in our test data set to within 12 m. This performance is two orders of magnitude better than PoseNet, a state-of-the-art neural network that, when trained on a large corpus of images in an area, can estimate the pose of a single image. Finally, we show that the camera's inferred position and intrinsic parameters can help design a number of virtual sensors, all of which are reasonably accurate.
BibTeX
@misc{ghosh2020localizing,
  author = {Pradipta Ghosh and Xiaochen Liu and Hang Qiu and Marcos Augusto M. Vieira and Gaurav S. Sukhatme and Ramesh Govindan},
  title = {{On Localizing a Camera from a Single Image}},
  year = {2020},
  eprint = {2003.10664},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2003.10664},
}
Figure