Multisensory Connections of Novel Linguistic Stimuli in Japanese as a Native Language and Referential Tastes

Yan Yan, Yutao Yang, Misa Ando, Xinyi Liu, Toshimune KambaraEuropean 2021 · DOI

TL;DR

This study investigates how healthy adults learn new words when paired with taste stimuli, finding that learning words alone is more efficient than learning them with tastes. However, subjective taste experiences, like sweetness, can still influence how well these words are recalled later.

What Problem Does This Paper Address?

While it's known that people with autism or synesthesia have strong links between words and tastes, less is understood about how these multisensory connections affect learning in healthy individuals. This paper aims to fill that gap by examining the associative learning of novel Japanese words and taste stimuli (candies/chocolates) in healthy adults, specifically looking at how this influences memory recall and recognition. The researchers want to understand if pairing words with tastes aids or hinders learning compared to learning words in isolation.

Methods

The study employed a within-subjects design with healthy Japanese native speakers. The experiment involved four phases:

  1. Evaluation Phase: Participants rated 20 different gustatory stimuli (candies/chocolates) on ten semantic differential scales (e.g., sweetness, hardness, familiarity, preference).
  2. Learning Phase: Participants learned either novel linguistic forms paired with gustatory stimuli (G condition) or novel linguistic forms alone (W condition).
  3. Recognition Memory Phase: Participants identified which linguistic forms they had encountered during the learning phase.
  4. Free Recall Phase: Participants wrote down all the linguistic forms they remembered from the learning phase.

The researchers used Superlab software for stimulus presentation and performance recording. Statistical analyses included paired t-tests to compare performance between the G and W conditions, and correlation analyses to explore relationships between subjective taste evaluations and memory performance.

Datasets

The study utilized a dataset of 80 meaningless Japanese novel words and 20 gustatory stimuli (candies and chocolates). The novel words were selected from a previous study ([37]) based on their meaningfulness and non-association values. The gustatory stimuli were sourced from local supermarkets. No new benchmarks were introduced; the focus was on existing word and taste stimuli.

Key Insights and Ideas

  • Word-only learning is more efficient: For healthy individuals learning over a single day, learning novel words in isolation (W condition) leads to better recognition memory than learning them paired with taste stimuli (G condition). This suggests that the added cognitive load of processing gustatory information can detract from pure word learning.
  • Taste influences recall, not just recognition: While recognition memory was better for words learned alone, the subjective experience of taste (specifically sweetness) negatively correlated with the free recall of words paired with those tastes. This implies that even if the initial learning is less efficient, the sensory experience can still impact later retrieval.
  • Individual taste perception matters: The study highlights that personal evaluations of gustatory features (like sweetness) can play a role in how well associated words are remembered, suggesting a deeper connection between sensory experience and linguistic memory than just simple association.
  • Consistency in memory performance: Accuracy rates in the recognition memory phase for the G condition positively correlated with performance in the free recall phase for G. This indicates that individuals who are good at recognizing words associated with tastes are also likely to recall them better.

Key Results

  • Recognition Memory: Participants showed significantly higher recognition memory scores for novel linguistic forms learned alone (W) compared to those learned with gustatory stimuli (G). Specifically, the hit rate for W was higher than for G (t(15) = 3.17, p < 0.01, d = 0.79). Corrected recognition scores also favored W over G (t(15) = 2.95, p < 0.05, d = 0.74).
  • Response Times: There was no significant difference in response times between the G and W conditions during the recognition memory phase (t(15) = 0.46, p = 0.65, d = 0.12).
  • Free Recall: No significant difference was found in the proportion of words recalled between the G and W conditions (t(12) = 1.09, p = 0.30, d = 0.30).
  • Correlations: Subjective evaluation of sweetness negatively correlated with the free recall score of linguistic forms connected to those stimuli. Accuracy rates in the recognition memory phase for G positively correlated with free recall phase performance for G.

Limitations and Future Work

  • Sample Size and COVID-19: The study was limited by the inability to collect additional samples due to the COVID-19 pandemic. The authors suggest future studies should increase the sample size.
  • Single-Day Learning: The experiment focused on learning within a single day. Future research could explore the effects of longer-term learning or spaced repetition.
  • Sound Symbolism: The study did not investigate the effects of sound-symbolic (non-arbitrary) connections between linguistic forms and gustatory stimuli. Future work could explore how non-arbitrary links (e.g., specific sounds associated with certain tastes) might influence learning.
  • Verbal Working Memory: The role of verbal working memory in associative learning was not explicitly examined but is suggested as an area for future investigation.
  • Generalizability: While the findings are relevant for healthy individuals, the authors note that people with sensory-related disabilities might exhibit different learning patterns, warranting further comparative studies.

Summary updated

Abstract

Previous findings have shown essential connections between linguistic and gustatory stimuli for people with autism or lexical gustatory synesthesia. We examined the associative learning of novel linguistic forms in Japanese as a native language and tastes (candies and chocolates) for healthy people. Healthy subjects performed four phases: (a) evaluation phase of gustatory features; (b) learning phases of novel linguistic form and gustatory stimulus pairs (G) or novel word forms (W); (c) recognition memory phases linked with G and W; and (d) free recall phase for G and W. In the recognition memory phases, the performance scores of W were higher than those of G, while there was no significant difference between response times of G and W. Additionally, no difference between recall performances in G and W was also shown. A subjective evaluation of gustatory features (sweetness) negatively correlated with the recall score for linguistic forms connected to the gustatory feature, whereas the accuracy rates of the recognition memory phase in G positively correlated with those of the free recall phase in G. Although learning of novel linguistic forms is more efficient than learning of the relationships between novel linguistic forms and tastes, gustatory features influence the free recall performances of linguistic forms linked with the tastes. These results may contribute to future applications to word learning not just for patients, but also healthy people.
BibTeX
@inproceedings{yan2021multisensory,
  author = {Yan Yan and Yutao Yang and Misa Ando and Xinyi Liu and Toshimune Kambara},
  title = {{Multisensory Connections of Novel Linguistic Stimuli in Japanese as a Native Language and Referential Tastes}},
  booktitle = {European},
  year = {2021},
  doi = {10.3390/ejihpe11030074},
}
Figure