Photosensor-Based Latency Measurement System for Head-Mounted Displays

Min-Woo Seo, Song-Woo Choi, Sang-Lyn Lee, Eui Yeol Oh, Jong-Sang Baek, Suk-Ju Kang2017 · DOI

TL;DR

This paper introduces a novel, highly accurate system for measuring the "motion-to-photon" latency in Head-Mounted Displays (HMDs). By precisely modeling physical head movement and detecting subtle changes in display luminance, their system overcomes limitations of existing methods, enabling more reliable latency analysis crucial for reducing VR-induced motion sickness.

What Problem Does This Paper Address?

Virtual Reality (VR) experiences, particularly those delivered via Head-Mounted Displays (HMDs), are rapidly gaining popularity due to their immersive potential. However, a significant hurdle to widespread adoption and user comfort is "motion sickness" and dizziness, largely attributed to the delay between a user's physical head movement and the corresponding update on the HMD's display. This delay, known as "motion-to-photon latency," arises from various stages in the VR pipeline, including sensor input, processing, and display rendering. Existing measurement systems often lack accuracy, fail to adequately account for actual physical head movement, or are incompatible with different HMD models. This paper aims to address these shortcomings by proposing a more robust and versatile latency measurement system.

Methods

The core of the proposed system is a photosensor-based latency measurement system that integrates several key components:

  • Head Position Model-Based Rotary Platform: This platform is designed to precisely mimic human head movements. It utilizes high-accuracy DC motors and encoders to achieve precise control over yaw and pitch rotations, simulating realistic head motion. The system can replicate movements within typical human constraints (±50° yaw, ±40° pitch) and achieve high angular velocities.
  • Pixel Luminance Change Detector: This component is responsible for measuring the actual visual output of the HMD. It employs four photosensors positioned in front of the HMD's display panel. These sensors detect changes in pixel luminance caused by the rendered image updates. To enhance sensitivity to subtle luminance changes, a "virtual lens technique" is employed, which effectively magnifies the pixel luminance, akin to using a magnifying glass. This allows for the detection of low-level luminance changes that might otherwise be missed.
  • Signal Analysis and Calculation Modules: A control PC orchestrates the entire process. It receives signals from the encoders (indicating physical movement) and the photosensors (indicating display luminance changes). Sophisticated signal processing, including a thresholding technique to remove noise, is used to accurately identify the precise moment of physical movement and the corresponding moment of luminance change. The difference between these two time points is calculated as the motion-to-photon latency.

Datasets

Not discussed in the paper. The experiments were conducted using a commercial HMD (Oculus Rift DK2) and custom-generated virtual environments with varying rendering workloads.

Key Insights and Ideas

  • Physical Head Movement is Crucial: The paper strongly emphasizes that accurate latency measurement must consider actual physical head movement, not just input signals. Existing systems that ignore this are inherently flawed.
  • Virtual Lens for Enhanced Sensitivity: The "virtual lens technique" is a clever approach to overcome the limitations of photosensors in detecting very small luminance changes on display pixels. This is critical for accurate measurement, especially with high-performance HMDs that aim for very low latency.
  • Workload Significantly Impacts Latency: The experiments clearly demonstrate that increasing the rendering workload (e.g., by increasing the number of polygons in a scene) dramatically increases motion-to-photon latency. This highlights the importance of optimizing rendering performance for a smooth VR experience.
  • Versatility is Key: The proposed system's design, particularly the use of standard components and its focus on measuring physical movement and display output, makes it adaptable to various commercially available HMDs, unlike some previous, more proprietary solutions.

Key Results

  • The proposed system achieved accurate latency measurements for a commercial HMD, with measured latencies up to 47.05 ms.
  • The system demonstrated high reliability, with low standard deviations in latency measurements across different rotation angles (yaw and pitch).
  • When the rendering workload was increased, the motion-to-photon latency saw a significant rise, reaching up to 381.17 ms. This underscores the system's ability to capture latency variations under different performance conditions.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the current system measures latency for individual frames. Future work aims to extend this to measure latency across successive frames.
  • While the system is designed for compatibility, the implementation details might require adjustments for specific HMDs.
  • Future Work:
    • Develop a complete system with a graphical user interface (GUI) for easier operation and visualization.
  • Focus on tracking and expressing head orientation by analyzing changes in image intensity levels within the HMD.
  • Investigate measuring motion-to-photon latency in successive frames.

Summary updated

BibTeX
@misc{seo2017photosensor,
  author = {Min-Woo Seo and Song-Woo Choi and Sang-Lyn Lee and Eui Yeol Oh and Jong-Sang Baek and Suk-Ju Kang},
  title = {{Photosensor-Based Latency Measurement System for Head-Mounted Displays}},
  year = {2017},
  doi = {10.3390/s17051112},
}

Loading related content...

Figure