Abstract
Extended Reality (XR) experiences involve interactions between users, the real world, and virtual content. A key step to enable these experiences is the XR headset sensing and estimating the user's pose in order to accurately place and render virtual content in the real world. XR headsets use multiple sensors (e.g., cameras, inertial measurement unit) to perform pose estimation and improve its robustness, but this provides an attack surface for adversaries to interfere with the pose estimation process. In this paper, we create and study the effects of acoustic attacks that create false signals in the inertial measurement unit (IMU) on XR headsets, leading to adverse downstream effects on XR applications. We generate resonant acoustic signals on a HoloLens 2 and measure the resulting perturbations in the IMU readings, and also demonstrate both fine-grained and coarse attacks on the popular ORB-SLAM3 and an open-source XR system (ILLIXR). With the knowledge gleaned from attacking these open-source frameworks, we demonstrate four end-to-end proof-of-concept attacks on a HoloLens 2: manipulating user input, clickjacking, zone invasion, and denial of user interaction. Our experiments show that current commercial XR headsets are susceptible to acoustic attacks, raising concerns for their security.
Introduction: The Hardware-Level Attack Surface
Extended Reality (XR) headsets rely on Simultaneous Localization and Mapping (SLAM) to navigate physical environments. This process fuses visual data from cameras with high-frequency readings from an Inertial Measurement Unit (IMU), comprising accelerometers and gyroscopes. While software vulnerabilities are frequently scrutinized, the physical sensors themselves present a critical "hardware-level" attack surface.
Most modern IMUs utilize Micro-Electro-Mechanical Systems (MEMS), which consist of microscopic physical components like springs and masses. This research introduces "Siren Song," an out-of-band attack that leverages sound waves to physically vibrate these MEMS components at their resonant frequencies. By injecting "fake" motion data directly into the hardware pipeline, an attacker can manipulate the XR system's perception of reality without needing to compromise its software.
Methodology: Profiling and Simulating Acoustic Interference
The researchers employed a multi-stage approach to validate this threat:
- Acoustic Profiling: They conducted frequency sweeps (2-30 kHz) on commercial hardware, including the Microsoft HoloLens 2 and Meta Quest 3, to identify specific resonant frequencies that induce sensor spikes.
- Modeling & Simulation: Using data from hardware profiling, they developed a Gaussian Mixture Model (GMM) to simulate realistic sensor perturbations. These were applied to open-source SLAM frameworks like ORB-SLAM3 and ILLIXR/OpenVINS to observe how different algorithms handle corrupted inertial data.
- Physical Proof-of-Concept: To simulate real-world usage, a HoloLens 2 was mounted on a remote-controlled car. A portable speaker was then used to launch attacks, focusing on the "Snapback" effect—a state where the SLAM system, detecting inconsistent data, resets the user's coordinates to the origin.
The study utilized standard industry datasets, such as the EuRoC MAV dataset, alongside custom traces recorded with a RealSense D435i camera and HoloLens 2 to evaluate the attack's impact across various scenarios.
Key Insights: Exploiting System Stability
A primary finding is the "Snapback" vulnerability, which ironically exploits a security and stability feature. When SLAM systems like ORB-SLAM3 detect a significant discrepancy between IMU and camera data, they assume tracking has failed and reset the pose to (0,0,0). An attacker can trigger this reset intentionally to displace virtual objects relative to the user.
The impact is highly algorithm-dependent. While ORB-SLAM3 "snaps back," OpenVINS (used in ILLIXR) tends to drift toward infinity. Furthermore, the attack is most effective when the headset is in motion. Stationary headsets can more easily filter out acoustic noise as an outlier, but during movement, the "fake" signals blend with legitimate inertial data, making detection significantly harder.
Results and Attack Scenarios
The HoloLens 2 was found to be vulnerable at 2.65 kHz (accelerometer) and 17.7 kHz (gyroscope). In real-world tests, the researchers achieved a success rate exceeding 90% in triggering the snapback effect using an 85dB sound source.
These vulnerabilities enable several novel attack scenarios:
- Manipulating User Input: Forcing position resets in applications (e.g., racing games) to disrupt progress.
- Clickjacking: Inducing drift in virtual interfaces, such as keyboards, to trick users into incorrect inputs.
- Denial of Interaction: Using the snapback effect to position virtual "blocking" objects permanently in the user's field of view.
- Secure Zone Invasion: Manipulating coordinate systems to move unauthorized virtual objects into a user's private "secure zone."
Conclusion
While current attacks can trigger resets and drifts, achieving fine-grained "steering" remains a challenge for future work, requiring more precise control over IMU phase and amplitude. Potential mitigations include improved physical shielding or the implementation of Low-Pass Filters (LPFs) in sensor amplifiers, though these are difficult to retrofit onto existing commercial devices. The study highlights a significant security gap in current XR hardware that necessitates further attention as these devices become more prevalent.
Summary updated