Abstract
This study demonstrated performance impairments in patients with epilepsy (PWE) that may reflect dysfunction of nonverbal semantic memory circuits, such as seizure onset zones overlapping with key semantic regions (e.g., anterior temporal lobe). The Visual-based Semantic Association Task (ViSAT) paradigm avoids language-based confounds, is repeatable for longitudinal studies, captures detailed behavioral data, and is open-source. We propose it as a strong alternative for clinical and research assessment of nonverbal semantic memory.
Introduction
Semantic memory is our internal library of general knowledge, allowing us to associate concepts like "cherry" with "pie" or "space shuttle" with "moon." In clinical neurology, assessing this function is vital because many conditions, particularly epilepsy, affect the anterior temporal lobe—a critical hub for semantic processing.
However, traditional assessments face significant hurdles. Most rely on verbal naming, making it difficult to distinguish between a loss of the concept and a language deficit (aphasia). Furthermore, legacy nonverbal tests like the Pyramids & Palm Trees (PPT) test are often proprietary, use limited black-and-white drawings, and lack the ability to track reaction times or account for the subjective nature of human associations.
The ViSAT Paradigm
To address these limitations, the researchers developed the Visual-based Semantic Association Task (ViSAT). This open-source tool consists of 100 unique trials featuring real-life color photographs. In each trial, participants view a stimulus image and must select the most related image from four choices.
To ensure the task measures meaning rather than simple visual matching, the team used a deep learning model (ResNet-18) to confirm that correct answers were not more visually similar to the stimulus than distractors. The task was administered via a MATLAB-based interface that separately timed the perception stage (viewing the target) and the association stage (choosing the answer).
Validation and Consensus-Based Scoring
Unlike traditional tests that assume a single binary "correct" answer, ViSAT uses consensus-based scoring derived from crowdsourced data. This treats semantic association as a probabilistic human agreement. Validation against established benchmarks showed that ViSAT achieved a consensus of over 90% in the vast majority of its trials, significantly outperforming older instruments like the Camels & Cactus Test (CCT).
Clinical Findings in Epilepsy
The study compared healthy controls against a group of patients with epilepsy (PWE). The results revealed specific deficits in the clinical group:
- Accuracy: PWE had significantly lower accuracy (91.4%) compared to controls (94.4%).
- The "Semantic Delay": While PWE were slower overall, their delay was disproportionately larger during the association phase (choosing the answer) compared to the perception phase. This suggests a specific bottleneck in semantic retrieval rather than a general slowing of visual processing.
- Anatomical Correlation: Patients with seizure onset zones in the temporal lobe were particularly affected, reinforcing the role of these regions in nonverbal semantic memory.
Conclusion and Future Directions
ViSAT provides a modernized, culturally neutral, and rigorous method for assessing semantic memory. While the current study's sample size limits definitive mapping of specific brain regions to error types, the tool's open-source nature allows for wide adoption and future cross-cultural validation. As a nonverbal, repeatable paradigm, ViSAT is well-positioned for longitudinal tracking of cognitive health in patients with epilepsy and other neurological disorders.
Summary updated