High-Density Functional Near-Infrared Spectroscopy and Machine Learning for Visual Perception Quantification

Hongwei Xiao, Zhao Li, Yuting Zhou, Zhenhai Gao2023 · DOI

TL;DR

This paper introduces a novel approach to quantify human visual perception using high-density functional near-infrared spectroscopy (HfNIRS) combined with machine learning. By analyzing brain activity patterns (Hb, HbO2, Hbt) during simulated driving scenarios with varying visual stimuli, the study demonstrates that HfNIRS can effectively differentiate and quantify subjective visual experiences, achieving high classification accuracy.

What Problem Does This Paper Address?

Quantifying subjective human experiences like visual perception is a long-standing challenge, especially in applied fields like automotive design where understanding user responses to visual stimuli is crucial. Traditional methods often rely on subjective questionnaires, which can be unreliable. While functional near-infrared spectroscopy (fNIRS) offers a non-invasive way to measure brain activity related to visual processing, existing approaches often lack the spatial resolution or comprehensive analysis needed for precise quantification. This paper aims to bridge this gap by using high-density fNIRS (HfNIRS) and advanced machine learning techniques to objectively measure and quantify visual perception in a controlled, applied setting.

Methods

The study employs a multi-faceted approach:

  • High-Density fNIRS (HfNIRS): Unlike conventional fNIRS, HfNIRS utilizes a denser array of sensors to achieve higher spatial resolution and capture more detailed brain activity, particularly in the cerebral cortex.
  • Simulated Driving Environment: A realistic car driving simulator was used to present controlled visual stimuli, specifically varying the brightness and saturation of the car's operating interface.
  • Task Design: Participants engaged in visual tasks that involved answering questions about the layout, color, design, and information presented in the simulated scene.
  • Data Preprocessing and Channel Screening: Raw HfNIRS data (Hb, HbO2, Hbt) were normalized. Mutual information was then used to reduce the dimensionality of the data by selecting the most informative channels, mitigating the "curse of dimensionality."
  • Feature Extraction: A range of statistical and information-theoretic features (e.g., mean, variance, entropy) were extracted from the processed HfNIRS signals.
  • Machine Learning Classification: The K-Nearest Neighbors (KNN) algorithm was employed to classify different visual tasks and their corresponding brain activity patterns. This allowed for the quantification of visual perception based on the accuracy of task classification.
  • Correlation Analysis: The study correlated questionnaire scores with HfNIRS signal fluctuations to establish a link between subjective perception and objective brain activity.

Datasets

The study collected data from 29 volunteers. The dataset consists of:

  • High-density functional near-infrared spectroscopy (HfNIRS) data measuring Hb, HbO2, and Hbt concentrations.
  • Questionnaire data related to visual perception tasks (layout, design, color, information) within a simulated driving scenario.

No new benchmark datasets were introduced; rather, existing methods were applied to a newly designed experimental setup.

Key Insights and Ideas

  • HfNIRS as a Quantifiable Measure of Visual Perception: The core insight is that HfNIRS, when combined with sophisticated data analysis and machine learning, can move beyond simply detecting brain activity to quantifying subjective visual perception.
  • Dimensionality Reduction is Crucial: The paper highlights the importance of techniques like mutual information for channel screening. This is essential for handling the high-dimensional nature of HfNIRS data and improving model performance and interpretability.
  • Task-Specific Brain Signatures: Different visual tasks (layout, design, color, information) elicited distinct patterns in HfNIRS signals, suggesting that specific brain activity signatures can be linked to different aspects of visual perception.
  • Correlation Between Subjective Scores and Brain Activity: A significant finding is the direct correlation between higher questionnaire scores (indicating higher satisfaction or better perception) and more pronounced fluctuations in Hb, HbO2, and Hbt. This provides objective validation for subjective assessments.
  • Feasibility for Applied Design: The study demonstrates the potential of this approach for fields like automotive design, offering a way to objectively evaluate user responses to visual elements.

Key Results

  • High Classification Accuracy: The KNN classification model achieved an average accuracy of 96.3% ± 1.99% in classifying different visual tasks and their corresponding brain activity patterns.
  • Significant Differences in Brain Activity: Statistical analysis (p-values < 0.05) revealed significant differences in Hb, HbO2, and Hbt fluctuations between high-score and low-score groups for various visual tasks, particularly layout and design problems.
  • Correlation Strength: Higher questionnaire scores were consistently associated with more obvious, significant, and higher fluctuations in Hb, HbO2, and Hbt, reinforcing the link between subjective experience and objective brain response.
  • Channel Reduction Effectiveness: The mutual information-based channel reduction retained a subset of channels (e.g., 7 channels for layout problems) that were sufficient for accurate classification, demonstrating the efficiency of this approach.

Limitations and Future Work

  • Limitations:
    • Limited Experimental Pictures and Population: The study acknowledges that the range of visual stimuli (pictures) and the participant pool were limited, which might restrict the generalizability of the findings.
  • Partial Quantification: The authors state that the quantification of visual perception was only "partially achieved," suggesting room for further refinement.
  • Future Work:
    • Expand Visual Stimuli: Incorporate a wider variety of pictures with different colors and complexities for more accurate quantification.
  • Explore Channel Interactions: Further investigate the relationships and interactions between different HfNIRS channels.
  • Enhance Evaluation System: Develop the evaluation system further, particularly for the automotive industry, to validate its effectiveness in real-world applications.
  • Broader Populations and Features: Expand the study to include diverse working populations and explore more advanced depth features.
  • Multi-modal Fusion: Integrate other physiological signals (e.g., EEG, eye-tracking) for a more comprehensive quantification.
  • Alternative Classification Methods: Explore other machine learning algorithms to potentially achieve even more accurate results.

Summary updated

BibTeX
@misc{xiao2023high,
  author = {Hongwei Xiao and Zhao Li and Yuting Zhou and Zhenhai Gao},
  title = {{High-Density Functional Near-Infrared Spectroscopy and Machine Learning for Visual Perception Quantification}},
  year = {2023},
  doi = {10.3390/s23218696},
}

Loading related content...

Figure