Analyzing Attention Focus in the Cookie TheftPicture Description Task Using Word Alignment

Anna Favaro, Thomas Thebaud, Najim Dehak, Ankur Butala, Esther S. Oh, Laureano Moro-VelázquezAlzheimer 2024 · DOI

TL;DR

This paper uses speech timestamps from the Cookie Theft picture description task to analyze attention focus in individuals with Alzheimer's Disease (AD) and other neurological conditions. They found that AD patients exhibit fewer visits to "Areas of Interest" (AoIs) and spend less time on external inanimate objects, suggesting difficulties in directing and maintaining attention.

What Problem Does This Paper Address?

Visuospatial difficulties are a known symptom of Alzheimer's Disease (AD). Previous research using eye-tracking in static image searches has shown that individuals with dementia tend to fixate on fewer areas and for shorter durations. However, understanding how attention is directed and maintained dynamically through speech, especially in relation to visual stimuli, remains an open area. This paper aims to bridge this gap by inferring attention focus from speech patterns during a descriptive task, comparing individuals with AD to those with Parkinson's Disease (PD), Parkinson's Disease mimics (PDM), and healthy controls (CTLs).

Methods

The core innovation here is using automatic speech recognition (ASR) to extract word timestamps from spoken descriptions of the "Cookie Theft" picture. The WhisperX ASR model was employed for this. From these timestamps, several features related to attention focus were computed:

  • Time spent in AoIs: How long participants focused on specific visual elements.
  • Time to approach the first AoI: How quickly participants began engaging with the visual scene.
  • Number of visits to AoIs: How many times participants shifted their attention to different elements.
  • Transition time: The duration between shifts in attention.
  • Time elapsed between first and last AoI: The overall duration of engagement with the visual scene.

These features were calculated for all identified Areas of Interest (AoIs), categorized as animate, inanimate, or collectively. Statistical analysis, specifically non-parametric Mann-Whitney U tests with Benjamini-Hochberg correction for False Discovery Rate, was used to compare these features across the four experimental groups (AD, PD, PDM, CTL).

Datasets

The study utilized data from participants performing the "Cookie Theft" picture description task. The groups included:

  • Alzheimer's Disease (AD): n=20
  • Parkinson's Disease (PD): n=43
  • Parkinson's Disease mimics (PDM): n=16
  • Control (CTL): n=33

No new datasets were introduced; existing recordings of the Cookie Theft task were used.

Key Insights and Ideas

  • Speech as a proxy for attention: The paper demonstrates that temporal patterns in speech (word timestamps) can serve as a valuable, non-invasive proxy for inferring visual attention focus, complementing traditional eye-tracking methods.
  • AD-specific attention deficits: The findings highlight distinct attention patterns in AD, suggesting that difficulties are not just about what is looked at, but how attention is directed and sustained over time.
  • Distinguishing neurological conditions: The analysis reveals subtle differences in attention patterns between AD, PD, and PDM groups, hinting at the potential of this method for differential diagnosis or understanding disease progression.

Key Results

  • AD vs. CTL: Subjects with AD showed a significantly fewer number of visits to AoIs and spent less time on inanimate external AoIs compared to controls. They also exhibited greater variability in transition times between AoIs.
  • PDM vs. PD: The PDM group spent significantly more time on animate AoIs than the PD group. However, PDM participants made fewer visits to internal inanimate AoIs within the house compared to PD participants.

Limitations and Future Work

The authors acknowledge that this is a preliminary analysis. A significant limitation is the reliance solely on speech data. Future work will incorporate eye-tracking technology to directly correlate speech-derived attention focus with actual eye movements, providing a more comprehensive understanding of the relationship between visual attention and verbal description in different neurological conditions.

Summary updated

Abstract

Abstract Background Visuospatial problems are one of the possible signs of Alzheimer’s Disease (AD). Notably, various studies on eye movement in static image searches reveal that individuals with dementia fixate on fewer Areas of Interest (AoIs) for a significantly shorter duration than normal controls (CTLs). Building on prior works on speech and eye movement, in this study we examined word timestamps to infer each subject’s focus of attention while describing the Cookie Theft picture. Namely, we computed various interpretable features encoding attention focus and compared them across different neurological disorders. Method Cookie Theft Picture description tasks were recorded from subjects with AD (n=20), Parkinson’s Disease (PD) (n=43), Parkinson’s disease mimics (PDM) (n=16), and CTLs (n=33). First, speech transcriptions and word timestamps were generated using WhisperX automatic speech recognizer. The following features were then computed using word timestamps: time spent in the AoIs, time to approach the first AoI, number of visits to the AoIs, transition time from one AoI to another, and time elapsed between the first and last AoI mentioned. Features were computed considering all the animate AoIs, all the inanimate AoIs, and all AoIs collectively. Completed the feature extraction, non‐parametric pair‐wise Mann‐Whitney U tests of significance were conducted to determine whether there were significant differences between experimental groups for each feature. To control the False Discovery Rate, we applied the Benjamini–Hochberg correction. Result Results from the statistical analysis showed that subjects with AD reported a significantly fewer number of visits to the AoIs and a shorter amount of time spent on inanimate external AoIs than CTLs. The AD group also reported a greater variability in the transition time from one AoI to another. Moreover, the PDM group spent significantly more time (on average) than the PD group on animate AoIs but reported a fewer number of visits to inanimate AoIs located internally to the house than PDs. Conclusion Our preliminary analysis confirmed the difficulties of subjects with cognitive decline to voluntarily direct attention to central cues and maintain attention focus on given AoIs. Our future work will include eye‐tracking technology to analyze the relationship between attention focus from speech and eye movement.
BibTeX
@inproceedings{favaro2024analyzing,
  author = {Anna Favaro and Thomas Thebaud and Najim Dehak and Ankur Butala and Esther S. Oh and Laureano Moro-Velázquez},
  title = {{Analyzing Attention Focus in the Cookie TheftPicture Description Task Using Word Alignment}},
  booktitle = {Alzheimer},
  year = {2024},
  doi = {10.1002/alz.091687},
}
Figure