A Classification Method for Japanese Signs Using Manual Motion Descriptions

Hisahiro Adachi, Kazuo KamataCOLING 1994

TL;DR

This paper introduces a novel method for classifying Japanese Sign Language (JSL) signs by analyzing the similarity of their "Manual Motion Descriptions" (MMDs), which are textual descriptions of sign movements. By transforming these descriptions into feature vectors and calculating their longest common subsequence (LCS), the authors demonstrate a way to group semantically related signs, potentially aiding in sign language recognition and dictionary creation.

What Problem Does This Paper Address?

Accurately classifying and organizing sign language signs is crucial for developing sign language recognition systems, digital dictionaries, and translation tools. Traditional linguistic approaches to describing signs, while detailed, can be difficult to translate into computational representations. This makes it challenging to process large datasets of signs and identify subtle similarities or differences that are key to understanding sign language structure and meaning. The authors aim to bridge this gap by proposing a computational method that leverages textual descriptions of sign movements to establish a quantifiable similarity measure.

Methods

The core of the proposed method lies in transforming Manual Motion Descriptions (MMDs) into a format suitable for mathematical analysis.

  • MMD Characteristics: The authors note that MMDs, while in natural language, exhibit more constrained syntactic patterns and word usage than general sentences. They also highlight the role of kanji characters in conveying semantic groupings.
  • Feature Vector Transformation: MMDs are converted into n-dimensional feature vectors. This is achieved by constructing a finite state automaton that accepts the MMDs. The properties of these feature vectors are derived from the characters within the MMDs, effectively creating a bit vector representation.
  • Similarity Measure: The similarity between two signs is defined as the cosine of the angle between their corresponding feature vectors. This similarity is further operationalized by calculating the squared ratio of the Longest Common Subsequence (LCS) of the MMDs to the product of their lengths. The LCS is a standard string processing technique that finds the longest sequence of characters common to both strings, preserving their order.
  • Classification: The similarity measure, while reflexive and symmetric, is not strictly transitive. To address this, the authors introduce a transitive closure operation to create equivalence classes of signs. These classes, formed at different similarity thresholds, can be used to build hierarchical clusters, akin to a dendrogram, for sign classification.

Datasets

The study utilizes data from "The Illustrated Sign Dictionary" (Maruyama, 1984). The initial dataset contained 1,527 entries, which were merged to 1,514 after accounting for identical MMDs. For a specific experiment, a subset of 129 entries was used, focusing on two keywords: "mouth" (71 entries) and "lips" (58 entries).

Key Insights and Ideas

  • MMDs as a Computational Bridge: The paper demonstrates that textual descriptions of sign movements (MMDs) can be effectively used as a computational basis for sign similarity and classification, bypassing the need for complex visual analysis in the initial stages.
  • LCS for Sign Similarity: The Longest Common Subsequence (LCS) of MMDs provides a robust and computationally efficient measure of similarity between signs, capturing both the presence and order of descriptive elements.
  • Identifying Minimal Pairs: The similarity measure successfully identifies "minimal pairs" of signs – pairs that differ by a single semantic or motion component. This is crucial for understanding the building blocks of sign language and how subtle changes alter meaning (e.g., antonyms like "daughter" and "son," or synonyms like "sad" and "cry").
  • Iconic Motion and Meaning: The research highlights how iconic motions, even when applied to different concepts (e.g., the "razor" motion for "Yokohama" and "smooth"), can lead to shared sign families, underscoring the interplay between visual representation and semantic meaning.

Key Results

  • The proposed similarity measure based on LCS of MMDs was shown to be effective in grouping semantically related signs.
  • Experiments indicated that a significant number of sign pairs exhibited high similarity. For instance, out of 36 pairs, similarities greater than 0.8 were observed, and 570 pairs had similarities greater than 0.5.
  • The classification method successfully identified clusters of signs, such as the "RED-family" comprising 23 signs, all sharing a common MMD and semantic component related to the color red.
  • The method was able to distinguish between antonyms and synonyms based on their MMD similarities, suggesting its utility in identifying these crucial linguistic relationships.

Limitations and Future Work

  • Transitivity Issue: The similarity measure itself is not strictly transitive, requiring the introduction of a transitive closure operation for classification.
  • Future Work: The authors propose applying this similarity measure to sign language retrieval systems, particularly for unknown signs where the motion image can be represented by an MMD. They also suggest further exploration of the similarity among MMDs and signs in a Sign Electronic Dictionary (SED). The paper also notes that the "curve cos² θ" feature of the similarity measure allows for ignoring noise factors at lower similarity levels, focusing on significant sign families rather than detailed hierarchy structures.

Summary updated

BibTeX
@inproceedings{adachi1994classification,
  author = {Hisahiro Adachi and Kazuo Kamata},
  title = {{A Classification Method for Japanese Signs Using Manual Motion Descriptions}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1994},
}

Loading related content...

Figure