Saliency difference based objective evaluation method for a superimposed screen of the HUD with various background

Hailong Liu, Toshihiro Hiraoka, Takatsugu Hirayama, Dongmin KimarXiv 2019 · arXiv

Introduction

Head-Up Displays (HUDs) are increasingly common in modern vehicles, designed to enhance safety by projecting critical information—such as speed and navigation—directly onto the windshield. This allows drivers to maintain focus on the road. However, unlike aviation HUDs that typically operate against uniform backgrounds (sky), automotive HUDs must contend with a highly dynamic and chaotic visual environment.

The primary challenge is mutual interference. Because HUDs are transparent, their visual elements blend with the background. A red warning icon might become invisible against a car's brake lights, or a high-brightness graphic might inadvertently mask a pedestrian. Evaluating these visibility issues traditionally requires expensive and time-consuming subjective studies. This paper presents an objective, algorithmic method to quantify HUD visibility and distraction using saliency differences.

Methodology: The Difference Saliency Method

The core of the proposed approach is the Difference Saliency Method. Saliency refers to the "eye-catchingness" of a pixel based on its contrast, color, and orientation relative to its surroundings. By comparing the saliency of HUD elements in a controlled environment versus a real-world driving scene, the researchers can quantify interference.

Figure 1: Flowchart of the Difference Saliency Method, illustrating the calculation of distraction (p) and saliency reduction (m) indices.

As shown in Figure 1, the process involves five distinct steps:

  1. Capture: A "measured image" ($M$) is taken of the HUD in a real driving environment.
  2. Baseline: A "HUD image" ($H$) is created with the same graphics against a pure black background, representing maximum theoretical saliency.
  3. Saliency Mapping: Saliency maps ($M^S$ and $H^S$) are generated using Itti’s bottom-up model.
  4. Subtraction: The baseline saliency is subtracted from the real-world saliency to find the difference ($E = M^{S\_HUD} - H^S$).
  5. Index Calculation: Two key metrics are derived:
  6. $p$ (Plus value): Measures visual distraction. High values indicate the background is "leaking" through, creating visual noise.
  7. $m$ (Minus value): Measures saliency reduction. High values indicate the background is "muting" the HUD information, making it harder to perceive.

Experimental Setup and Dataset

To validate the method, the authors recorded five hours of original driving footage in Nagoya, Japan, using a GoPro HERO+. This footage captured a wide range of conditions, including daytime, evening, and night, as well as various glare sources like sunlight and oncoming headlights.

Figure 5: Experimental laboratory setup (left) and the various HUD color configurations (white, red, green, blue) used for testing (right).

In a laboratory setting (Figure 5), this footage was replayed on a 4K display behind a physical MAXWIN HUD-622 unit. This allowed for controlled, repeatable measurements across different HUD colors (white, red, green, and blue) and background scenarios.

Key Results and Insights

The experimental results confirmed several intuitive risks in HUD design with quantitative precision:

  • The Vulnerability of Red: Red HUD elements were found to be highly susceptible to saliency loss. In night-time tests, red elements had a minus value ($m$) of 0.043, significantly higher than white (0.020), green (0.023), or blue (0.021). This is because the driving environment is saturated with warm light sources like brake lights and street lamps that "wash out" red graphics.
  • Daytime Distraction: Daytime scenarios generally yielded higher "plus" values ($p$), indicating that complex daylight backgrounds provide more visual distraction than night scenes.
  • Transparency Risks: The study highlighted that excessive HUD saliency can act as a mask, potentially blocking the driver's perception of objects behind the display, such as pedestrians.
  • Design Efficiency: By utilizing this objective model, designers can iterate on interface layouts and color schemes much faster than through traditional human-in-the-loop simulator trials.

Conclusion and Future Work

While the Difference Saliency Method provides a robust objective framework, the authors identified several areas for future refinement. Currently, the model operates in the RGB color space; moving to LUV or LAB spaces would better align with human visual perception. Additionally, the authors plan to incorporate motion features to account for how dynamic background objects affect legibility. Finally, further validation against extensive subjective feedback will help fine-tune the relationship between the $p/m$ indices and actual driver performance.

Summary updated

BibTeX
@misc{liu2019saliency,
  author = {Hailong Liu and Toshihiro Hiraoka and Takatsugu Hirayama and Dongmin Kim},
  title = {{Saliency difference based objective evaluation method for a superimposed screen of the HUD with various background}},
  year = {2019},
  eprint = {1905.05601},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1905.05601},
}
Figure