Exploiting Sensor-Human Perception Gaps for Adversarial Attacks

18 concepts9 papers1 proposal

Traditional physical adversarial attacks rely on modifying objects in the visible spectrum—such as placing a carefully designed sticker on a stop sign or wearing custom eyeglass frames. While effective at tricking machine learning models, these attacks suffer from a fundamental flaw: they are easily detectable by human observers. To achieve true stealth, a growing body of research shifts the battlefield from digital pixel manipulation to the hardware-level discrepancies between human eyes and digital sensors.

Human vision and machine vision are built on entirely different physical mechanisms. By exploiting these gaps in the spectral, temporal, and polarization domains, attackers can inject high-magnitude physical perturbations that are completely invisible to humans but highly disruptive to computer vision systems.

The Spectral Gap: Infrared and Ultraviolet Light

The human eye is a biological transducer limited to a narrow band of electromagnetic radiation between roughly 400 and 700 nanometers. Silicon-based digital camera sensors, however, possess a natural sensitivity that extends far beyond these boundaries into the infrared (IR) and ultraviolet (UV) spectrums.

To improve camera performance in low-light or nighttime conditions, engineers designing autonomous vehicles and security systems often omit or modify standard infrared-cut filters. As documented by Sato et al. and Sato et al., this hardware design choice introduces a critical security vulnerability. An attacker can project structured IR laser reflections onto a traffic sign; to a human driver or passenger, the sign looks perfectly normal. However, the vehicle's camera captures the IR light as bright, high-contrast visual features, injecting malicious patterns directly into the image processing pipeline and causing catastrophic misclassifications.

[Physical Scene] ---> (Human Eye) ---------> Benign Traffic Sign
       |
       +-[Invisible IR/UV Laser] 
       |
       v
[Camera Sensor (No IR Filter)] ---> [Deep Learning Model] ---> Misclassification

This spectral exploitation works in the ultraviolet range as well. As surveyed by Sun et al. and Fang et al., non-visible light sources can easily bypass traditional defenses designed for the visible spectrum. For instance, Yuan et al. demonstrated that mounting invisible UV emitters on a hat allows an attacker to covertly deceive face recognition systems from multiple angles.

Alternatively, attackers can exploit chemical properties to bridge the spectral gap. Yuan et al. developed a method using latent fluorescent inks applied to traffic signs. These inks remain completely unnoticeable under normal conditions but absorb ultraviolet light from a remote source and emit visible-spectrum interference. This creates a latent attack vector that activates only when a specific light trigger is present, exploiting the fact that most autonomous perception systems lack visible-light filtering to block such emissions.

The Temporal Gap: Rolling Shutters and Flickering

The discrepancy between human and machine perception is not just a matter of what wavelengths are captured, but when they are captured. Human brains smooth out rapid changes in light through a phenomenon known as flicker fusion; if a light source blinks faster than approximately 60 Hz, we perceive it as a continuous, steady glow.

Digital cameras sample time differently. Rather than exposing the entire image sensor simultaneously (a global shutter), most modern digital sensors utilize a rolling shutter that records the scene row-by-row.

Light Intensity
  On  |---|   |---|   |---|   |---|   (High-frequency flickering LED)
  Off |   |---|   |---|   |---|   |--
----------------------------------------------------------------------
Sensor Exposure
  Row 1: Captures "On"  ---> Bright Stripe
  Row 2: Captures "Off" ---> Dark Stripe
  Row 3: Captures "On"  ---> Bright Stripe

Sayles et al. exploited this temporal gap by flickering LED lights at extremely high frequencies. Because the camera sensor samples different rows at different points in time, the rapid on-off cycle of the LED is translated into a spatial pattern of alternating bright and dark stripes across the captured image. To a human supervisor, the scene appears uniformly illuminated, but the deep learning model receives an image covered in high-frequency adversarial stripes that completely degrade its classification accuracy.

The Polarization and Geometric Gaps

Beyond color and timing, light possesses properties like polarization—the orientation of its directional wave oscillations. Human eyes are entirely blind to polarization states. However, specialized computer vision systems rely heavily on polarization data to eliminate glass reflections, map 3D shapes, or perform material classification.

Li et al. demonstrated that this domain is highly vulnerable to stealthy manipulation. By modifying a standard LCD projector to alter the polarization angles of projected light without changing its visible color or intensity, the authors injected high-magnitude adversarial perturbations into polarization-based vision systems. Because the human eye discards polarization information entirely, the attack remains completely invisible to the naked eye.

Even basic geometric and spatial differences can decouple human perception from machine inputs. Because vehicle-mounted cameras are physically positioned at different heights and angles than a human driver's line of sight, Sitawarin et al. showed that lenticular printing can be used to create "toxic signs." These signs exploit the angular discrepancy to display a benign corporate logo to human observers while feeding a malicious, high-confidence adversarial pattern to the vehicle's automated vision system.

Breaking the Imperceptibility Trade-off

In traditional digital adversarial machine learning, there is an inherent trade-off between attack efficacy and imperceptibility. To keep an attack hidden from humans, the mathematical perturbation (often bounded by an $L_\infty$ norm) must be kept incredibly small. In the physical world, small or faint visible perturbations are easily washed out by environmental noise, changing weather, or camera distance.

The core insight uniting these hardware-level optical attacks is that shifting the perturbation into a human-blind channel completely decouples attack magnitude from visual imperceptibility. An attacker can use a high-power infrared laser, a massive polarization shift, or an intense high-frequency flicker to create a high-magnitude, robust signal that guarantees a successful attack against a neural network. Yet, because the signal bypasses the biological constraints of human vision, its perceived magnitude to a human observer remains exactly zero. This fundamentally undermines human-in-the-loop safety systems, as human supervisors looking at the exact same physical scene will see absolutely nothing out of the ordinary.

Key Foundational Work

A survey of practical adversarial example attacks (2018) — contributed 2 concept elements to this direction.
DARTS: Deceiving Autonomous Cars with Toxic Signs (2018) — contributed 1 concept elements to this direction.
Invisible Perturbations: Physical Adversarial Examples Exploiting the Rolling Shutter Effect (2021) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 9 papers
  2. Version 1 · 2026-10-04 · 19 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_cross_modal_perception_gap_6933978e,
  title = {cross-modal perception gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-modal-perception-gap/rdr_6933978e78604d8385969353a4babc9d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_cross_modal_perception_gap_6933978e_v2,
  title = {cross-modal perception gap},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-modal-perception-gap/rdr_6933978e78604d8385969353a4babc9d/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake