Dual-Coding Theory and Connectionist Lexical Selection

Ye-Yi WangACL 1994 · DOI

Abstract

This paper explores the intersection of psycholinguistics and connectionist artificial intelligence, specifically investigating how human mental representations of language can inform machine translation systems. By implementing a connectionist model for lexical selection based on Paivio’s Bilingual Dual-Coding Theory, the research demonstrates that psychologically grounded architectures can achieve superior generalization on unseen data compared to standard back-propagation models.

Introduction

In the field of machine translation, lexical selection—the task of choosing the target-language word that most accurately reflects the meaning of a source-language word—remains a significant challenge. Historically, researchers faced a trade-off between rule-based interlingua approaches, which required intensive manual knowledge engineering, and early statistical approaches, which were learnable but often lacked structural linguistic depth. This paper proposes a middle ground: a learnable system grounded in a psychologically plausible model of bilingual mental representation.

Methods: The Bilingual Dual-Coding Model

The proposed system is built upon Paivio’s Bilingual Dual-Coding Theory. This psychological framework posits that bilingual individuals possess two distinct verbal systems (one for each language) containing "logogens" (word representations). These systems are interconnected and also linked to a shared "imagery system" containing "imagens" (mental images).

Bilingual Dual-Coding Theory model showing the interconnected L1 and L2 verbal systems and their shared imagery system.

To translate this theory into a computational model, the author utilizes Information-Theoretical Networks. Unlike standard neural networks that rely on iterative error correction (back-propagation), these networks assign weights based on the mutual information between units. This structure is considered "isomorphic" to the verbal systems in Dual-Coding Theory. The implementation involves constructing one network for each phrasal category (NP, VP, AP, etc.). During translation, the system activates the source-language head and its sub-structures as inputs, and the most active output unit in the target language is then selected as the translation.

Experimental Setup and Results

The model was evaluated using a dataset from the Conference Registration Telephony Conversations domain, consisting of approximately 500 English and 500 German words. The training set included 300 English/German functional structure (f-structure) pairs, while 154 sentential f-structures were reserved for testing generalization.

The performance of the information-theoretical network was compared against a standard back-propagation neural network. While the proposed model showed lower accuracy on the training data (84% vs. 92%), it demonstrated significantly better generalization on unseen test data, achieving 70% accuracy compared to 62% for the back-propagation model.

Discussion and Key Insights

The superior generalization of the dual-coding model suggests that mimicking human "associative processing"—the way humans link translation-equivalent terms through frequent co-occurrence—captures the interdependent nature of bilingual memory more effectively than simple pattern matching. Furthermore, because weights are calculated via mutual information in a single pass, the network learns extremely fast and can adapt incrementally. The model's reduced number of "free parameters" compared to back-propagation networks also helps prevent overfitting, making it more robust for real-world translation tasks where inputs are unpredictable.

Limitations and Future Directions

Despite its promise, the experiment was limited to a small, specialized vocabulary. Scaling this architecture to handle the massive and ambiguous vocabularies of open-domain translation remains a challenge. Additionally, while the theory includes an imagery system, the current computational implementation focuses primarily on the associative links between verbal systems. Future research could integrate visual or semantic features to more fully simulate the imagery system. Finally, the system's reliance on a separate structural analysis network means that errors in initial parsing can propagate to the lexical selection stage.

Summary updated

BibTeX
@inproceedings{wang1994dual,
  author = {Ye-Yi Wang},
  title = {{Dual-Coding Theory and Connectionist Lexical Selection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1994},
  doi = {10.3115/981732.981781},
}
Figure