Optical Lens Attack on Deep Learning Based Monocular Depth Estimation

Ce Zhou, Qiben Yan, Daniel Kent, Guangjing Wang, Ziqi Zhang, Hayder RadhaarXiv 2024 · OpenReview

Abstract

Monocular Depth Estimation (MDE) plays a crucial role in vision-based Autonomous Driving (AD) systems by utilizing a single-camera image to determine the depth of objects. In this paper, we investigate the security risks associated with MDE algorithms and introduce LensAttack, a physical attack that involves strategically placing optical lenses on the camera of an autonomous vehicle to manipulate perceived object depths. LensAttack encompasses two formats: concave lens attacks and convex lens attacks, each utilizing different optical properties to induce false depth perception. We construct a mathematical model of the attack, simulate it against state-of-the-art MDE models, and evaluate its real-world performance. Our results highlight a significant impact on depth estimation accuracy, potentially leading to catastrophic failures in AD systems.

1. Introduction

Deep learning-based MDE models have become sophisticated alternatives to expensive Lidar or stereo-vision setups in autonomous vehicles. However, because they operate on a single 2D image, these models rely on "implicit knowledge"—primarily the geometric relationship between an object's size in the frame and its actual distance. This paper identifies a fundamental security gap: by manipulating the perceived size of an object before light reaches the sensor, an attacker can bypass digital defenses. Unlike adversarial patches, LensAttack is a physical-layer attack targeting the sensor itself.

2. LensAttack Methodology

The core of LensAttack is rooted in classical optics, utilizing the thin lens equation ($1/f = 1/d_o + 1/d_i$) to model how an external lens alters the image formed on the camera sensor.

Optical principles of LensAttack: (a, b) real images from convex lenses, (c) virtual magnified image from a convex lens, and (d) virtual diminished image from a concave lens.

The attack utilizes two primary lens types:

  1. Concave Lens Attack: These lenses diverge light, making objects appear smaller (Figure d). To an MDE model, a smaller object implies a greater distance, which could prevent a vehicle from braking for an obstacle.
  2. Convex Lens Attack: These lenses can magnify objects when the object is within the focal length (Figure c). This makes objects appear larger and thus closer, potentially causing "phantom braking" or dangerous lane changes.

3. Experimental Evaluation

3.1 Simulation Results

Using the KITTI dataset, we evaluated LensAttack against three state-of-the-art MDE models: Monodepth2, Depth Hints, and Lite-mono. The effectiveness was measured using the Attack Error Rate (AER):

  • Concave Attacks: Achieved an average AER of 11.48%. In simulations, this could reduce perceived depth by as much as 13 meters.
  • Convex Attacks: Produced a significantly higher average AER of 29.84%.

3.2 Real-World Performance

Physical experiments were conducted using a modified Chevy Bolt and an iPhone 12 Pro Max in various driving environments, including normal driving and lane changing. The attack consistently tricked models into perceiving a car at 6m as being at 9m or 12m, demonstrating the practical feasibility of the threat in real driving environments.

4. Key Insights and Discussion

  • Size-Depth Vulnerability: MDE models are essentially "size-estimators." They are physically incapable of distinguishing between a small nearby object and a large distant one if the image size is identical.
  • Black-Box Universality: The attack requires zero knowledge of the underlying neural network architecture, affecting any vision-based model relying on perspective cues.
  • The Blur Trade-off: While external lenses introduce blur, the "size change" effect is so dominant that depth estimation still shifts significantly in the intended direction.

5. Conclusion

LensAttack exposes a critical vulnerability in monocular vision-based AD systems. While the physical attachment might be noticed by a human, current AI systems lack the robustness to detect such tampering. Future work must focus on blur detection algorithms and, most importantly, robust sensor fusion. Vehicles that cross-reference camera data with Lidar or Radar are much harder to fool, provided the fusion logic can identify and reject contradictory visual data.

Paper Citation: Ce Zhou, Qiben Yan, Daniel Kent, Guangjing Wang, Ziqi Zhang, and Hayder Radha. "Optical Lens Attack on Deep Learning Based Monocular Depth Estimation." arXiv:2409.17376v1 [cs.CR], 2024.

Summary updated

Abstract

Monocular Depth Estimation (MDE) plays a crucial role in vision-based Autonomous Driving (AD) systems. It utilizes a single-camera image to determine the depth of objects, facilitating driving decisions such as braking a few meters in front of a detected obstacle or changing lanes to avoid collision. In this paper, we investigate the security risks associated with monocular vision-based depth estimation algorithms utilized by AD systems. By exploiting the vulnerabilities of MDE and the principles of optical lenses, we introduce LensAttack, a physical attack that involves strategically placing optical lenses on the camera of an autonomous vehicle to manipulate the perceived object depths. LensAttack encompasses two attack formats: concave lens attack and convex lens attack, each utilizing different optical lenses to induce false depth perception. We begin by constructing a mathematical model of our attack, incorporating various attack parameters. Subsequently, we simulate the attack and evaluate its real-world performance in driving scenarios to demonstrate its effect on state-of-the-art MDE models. The results highlight the significant impact of LensAttack on the accuracy of depth estimation in AD systems.
BibTeX
@misc{zhou2024optical,
  author = {Ce Zhou and Qiben Yan and Daniel Kent and Guangjing Wang and Ziqi Zhang and Hayder Radha},
  title = {{Optical Lens Attack on Deep Learning Based Monocular Depth Estimation}},
  year = {2024},
  eprint = {2409.17376},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.17376},
}
Figure