An Orientation Sensor-Based Head Tracking System for Driver Behaviour Monitoring

Chen Lv (吕辰), Yifan Zhao, Lorenz Görne, Iek-Man Yuen, Dongpu Cao, Mark Sullman, Daniel J. Auger, Chen Lv, Huaji Wang, Rebecca Matthias, Lee Skrypchuk, Alexandros Mouzakitis

TL;DR

This paper introduces a novel head tracking system using twin orientation sensors to monitor driver behavior in Level 3 autonomous vehicles. Unlike camera-based systems, this sensor-based approach offers a wider range of motion detection, crucial for understanding driver engagement in non-driving activities (NDAs) and ensuring safe transitions back to driving.

What Problem Does This Paper Address?

As autonomous driving technology advances towards Level 3 (highly automated driving), drivers will have more opportunities to engage in non-driving activities (NDAs). However, legislation currently restricts this, and a critical challenge for future HAD systems is monitoring driver attention and readiness to retake control. Existing camera-based head tracking systems, while common, have limitations: they struggle with varying lighting conditions, face occlusions (e.g., by accessories), and have a restricted field of view for head movement, typically failing beyond ±20° in nodding and shaking axes. This limits their ability to accurately capture the full range of head movements associated with diverse NDAs, such as chatting with a passenger or using a smartphone. The paper aims to address this by developing a more robust and wider-range head tracking system.

Methods

The core of the proposed system is a pair of orientation sensors (Bosch BNO 055) integrated into a head strap. One sensor is worn by the driver to measure head movement, while the second, identical sensor is mounted to measure vehicle movement. This dual-sensor approach is key: by subtracting the vehicle's motion from the head's absolute motion, the system can isolate and accurately measure the driver's head movements.

The BNO 055 is a 9-axis absolute orientation sensor that fuses accelerometer, gyroscope, and magnetometer data. It provides high-fidelity navigation data and compensates for temperature effects. Sensor fusion, using a Kalman filter, is employed to combine gyroscope data (for rotation speed) with accelerometer data (for angle) to achieve accurate rotation angle measurements and mitigate drift. A custom Python-based software was developed for calibration, data display, and export, designed to be lightweight to run alongside a camera-based system.

For comparison, a state-of-the-art camera-based head tracking system was also implemented, utilizing face detection, facial landmark detection (using CLNF), and 3D projection.

Datasets

The paper does not explicitly mention training datasets. Instead, it focuses on evaluating the proposed system's accuracy and performance through a series of experiments:

  • Static Platform Validation: Using a Kuka welding robot to precisely control head orientation.
  • In-house Testing: Comparing the sensor-based system with the camera-based system in a controlled environment simulating a vehicle.
  • On-Road Testing: Evaluating the system in a real vehicle environment.
  • NDA Case Study: Assessing the system's ability to characterize driver behavior during specific NDAs (chatting and smartphone use) in a static vehicle.

Key Insights and Ideas

  • Dual-Sensor Fusion for Vehicle Independence: The most significant innovation is using a second sensor to track vehicle motion, allowing the system to isolate and accurately measure head movements independently of the vehicle's dynamics. This is crucial for real-world driving scenarios.
  • Expanded Range of Motion Detection: The sensor-based system demonstrably captures head movements beyond the ±20° limitation of typical camera-based systems, reaching angles up to 50° in shaking and nodding. This is vital for monitoring a wider range of NDAs.
  • Robustness to Occlusion and Lighting: Unlike camera-based systems, the orientation sensors are not affected by facial occlusions or varying lighting conditions, making them more reliable in diverse driving environments.
  • Characterizing NDAs: The case study effectively shows how head movement patterns (shaking and nodding) can differentiate between distinct NDAs like "chatting with a passenger" and "using a smartphone."

Key Results

  • Accuracy: On a static platform, the average error for nodding and shaking axes was less than 0.4°, with rolling at approximately 1.5°.
  • Comparison (In-house): The sensor-based system showed very close agreement with the camera-based system for shaking and nodding (average difference < 2°). However, the camera system failed beyond ±20° angles.
  • Comparison (On-Road): The average difference between systems increased on the road (e.g., 4.9° for shaking) due to reference device errors on a moving platform, but the sensor system still outperformed the camera system for large movements.
  • NDA Case Study: The system successfully differentiated between "chatting" (significant shaking towards the passenger, wider range) and "using a smartphone" (more consistent, near-zero shaking, significant nodding downwards). The paper highlights that the camera system would miss over 40% of the time for shaking during chatting and nodding during smartphone use due to angle limitations.

Limitations and Future Work

  • Rolling Axis Accuracy: The paper acknowledges that the rolling axis measurement has relatively lower accuracy (average error ~1.5°) compared to shaking and nodding. This limits its utility for NDA characterization.
  • Calibration: While a calibration process is described, the paper suggests that automating the calibration process would be beneficial.
  • Future Work: The authors suggest improving the accuracy of the rolling measurement and automating the calibration process. Further research could also explore integrating this system with other sensors (e.g., eye-tracking, microphones) for a more comprehensive driver monitoring solution. The findings are also intended to inform the design of future vehicle interfaces and the placement of non-contact sensors for real-world applications.

Summary updated

BibTeX
@article{2017orientation,
  author = {Chen Lv (吕辰) and Yifan Zhao and Lorenz Görne and Iek-Man Yuen and Dongpu Cao and Mark Sullman and Daniel J. Auger and Chen Lv and Huaji Wang and Rebecca Matthias and Lee Skrypchuk and Alexandros Mouzakitis},
  title = {{An Orientation Sensor-Based Head Tracking System for Driver Behaviour Monitoring}},
  journal = {Sensors},
  year = {2017},
  doi = {10.3390/s17112692},
}
Figure